App authentication woes on SharePoint (Token request failed. The remote server returned an error: (401) Unauthorized.)

This post was most recently updated on January 17th, 2023.

This article explains how to get rid of sudden and unexplainable 401 Access Denied errors when trying to authenticate against a fairly fresh Microsoft 365 / SharePoint Online tenant. This issue seems to be caused by a long-ish project to finally retire ACS – Azure Access Control service) on SharePoint (it’s retired everywhere else already!)

Note: This might still be an updating story, as the situation with ACS is definitely… Developing. Yeah, let’s call it that. It’s a developing situation.

Problem

In the beginning of September (2020), a lot of threads, chats and tickets started popping up about apps and scripts suddenly failing to authenticate against SharePoint Online. It seemed somewhat widespread and just worsening.

One way to reproduce this issue is to use app-only authentication in OfficeDevPnP’s AuthenticationManager:

var authman = new OfficeDevPnP.Core.AuthenticationManager();
using (Microsoft.SharePoint.Client.ClientContext ctx = authman .GetAppOnlyAuthenticatedContext(appUrl, clientId, clientSecret))
{
  Web web = ctx.Web;
  context.Load(web, w => w.Id, w => w.Title);
  context.ExecuteQueryRetry();
}

The very low-key and non-descriptive error was simply:

"The remote server returned an error: (401) Unauthorized."

Sometimes with an additional insult to injury:

{"error":"invalid_request","error_description":"Token type is not allowed."}

The weirdest part? The same code or script might work just fine for other tenants. The only difference should be the age of the tenant you’re authenticating against – it should be provisioned during or after August 2020.

Reason

This has been a long time coming I suppose, but Microsoft is pushing users away from ACS and using Client Id & Client Secret -combo to authenticate against Microsoft 365. Some time during 2020, Microsoft added a new tenant-level property called “DisableCustomAppAuthentication” to SharePoint Online. This property was first surfaced in the August 2020 release of SharePoint Online CSOM. This release is available as a NuGet package with version 16.1.20412.12000.

This property should be pretty useful for a gradual move away from ACS – your organization can approach whatever deadline Microsoft sets for disabling ACS completely by “soft-disabling” it first with this property, and seeing if something breaks. Furthermore, disabling ACS is definitely a security improvement, as leaking a client id and client secret might’ve lead to anyone being able to access the tenant from anyone with zero oversight and very bad governance in general.

However, this is soiled pretty badly by the weird rollout. Unfortunately, someone at Microsoft made the decision to set this property to be true by default, and it’ll affect any tenants provisioned after sometime late August, 2020.

That’ll break a lot of custom functionality like apps or PowerShell scripts that work on any older tenants.

And obviously, the whole security aspect is also completely destroyed by the fact that since you’ll now have to use Azure AD, which doesn’t contain granular, site-level permissions for app authentication. At all. So any app registration either gets everything, or nothing.

With ACS, you could assign permissions on a per- site collection level.

So much for security improvements!

But at least good folks at Microsoft seem to be aware of the issues with this default setting:

Solution

Well, essentially, you’ve got 2 options. Let me explain them below!

… aaand you should be done! Until it breaks again.

References

• Credit where credit is due:
  • My colleague Jouni Pohjolainen brought this one to my attention (wish he had a blog, would make it easier to attribute stuff like this! 😄)
  • Gautam, another one of my colleagues (is there a pattern here?) pointed out I was missing the (newly available) switch PnP commandlets have to enable ACS – thanks!
• Other people having this issue:
  • SharePoint App-Only Add-ins throwing 401 Unauthorized on newly created O365 tenants
    • Thread on Microsoft Q&A
  • SharePoint Online authorization issue ‘Token type is not allowed’
    • https://sharepoint.stackexchange.com/questions/284402/sharepoint-online-authorization-issue-token-type-is-not-allowed/284734
  • People enjoying the (well, partially) same issue on GitHub:
    • https://github.com/SharePoint/sp-dev-docs/issues/6903

Also worthwhile to see this Twitter conversation by my colleague Vardhaman:

So interesting new development for SharePoint App-Only permissions. The Tenant.DisableCustomAppAuthentication property introduced in latest version of CSOM can be used to disable granting permissions through appregnew.aspx/appinv.aspx https://t.co/Rog1ZhhmUT

— Vardhaman Deshpande (@vrdmn) September 21, 2020

• Author
• Recent Posts

Antti K. Koskela

Developer / Speaker / Consultant at Koskila / Precio Fishbone / Norppandalotti Software Co / Alter - Experience Ideas Ltd

Antti Koskela is a proud digital native nomadic millennial full stack developer (is that enough funny buzzwords? That's definitely enough funny buzzwords!), who works as Solutions Architect for Precio Fishbone, building delightful Digital Workplaces.

He's been a developer from 2004 (starting with PHP and Java), and he's been working on .NET projects, Azure, Office 365, SharePoint and a lot of other stuff. He's also Microsoft MVP for Azure.

This is his personal professional (e.g. professional, but definitely personal) blog.

Latest posts by Antti K. Koskela (see all)

• Merging on GitHub Actions fails with “could not read Username for ‘https://github.com’: No such device or address”? - December 24, 2024
• How to close the Sidebar in Microsoft Edge - December 17, 2024
• How to stop Surface Headphones from autoplaying audio when you place them on the desk? - December 10, 2024