SharePoint 2013 CSOM call from web part fails with 401 for all users

This issue has come up at least 3 times in the list of issues I deal with here in SharePoint Escalation Services Support team, so I thought I’ll write a post about this on how to resolve this problem.

Issue:

You’ve designed a simple Visual Web Part with the following code in button click event handler and deployed it to a SharePoint Server 2013 environment.

    1: protected void Button1_Click(object sender, EventArgs e)
    2: {
    3:   try
    4:     {
    5:       ClientContext clientContext = new ClientContext("https://team.contoso.com");
    6:       clientContext.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;
    7:       Web oWeb = clientContext.Web;
    8:       ListCollection oListCollection = oWeb.Lists;
    9:       clientContext.Load(oListCollection);
   10:       clientContext.ExecuteQuery();
   11:       lstLists.Items.Clear();
   12:       foreach (List oList in oListCollection)
   13:       {
   14:         lstLists.Items.Add(oList.Title);
   15:       }
   16:     }
   17:     catch (Exception ex)
   18:     {
   19:       lblMessage.Text = ex.Message;
   20:     }
   21: }

When you drop an instance of this web part on a page and click the button.  You’ll get the following error.

The remote server returned an error: (401) Unauthorized.

Cause:

If you look at the verbose ULS entries, you’ll see the following:

02/06/2014 06:20:18.74 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation General 8xfr Verbose PermissionMask check failed. asking for 0x08000000, have 0x00000000 1195719c-0516-2045-a82e-8077f54daf22

02/06/2014 06:20:18.74 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation General 8xfr Verbose PermissionMask check failed. asking for 0x00010000, have 0x00000000 1195719c-0516-2045-a82e-8077f54daf22

02/06/2014 06:20:18.74 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation General 8e2s Medium Unknown SPRequest error occurred. More information: 0x80070005 1195719c-0516-2045-a82e-8077f54daf22

02/06/2014 06:20:18.75 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation General aix9j High SPRequest.RenderFormDigest: UserPrincipalName=i:0).w|s-1-5-17, AppPrincipalName= ,bstrUrl=

https://team.contoso.com/\_vti\_bin/sites.asmx 1195719c-0516-2045-a82e-8077f54daf22

02/06/2014 06:20:18.75 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation General ai1wu Medium System.UnauthorizedAccessException: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)), StackTrace: at Microsoft.SharePoint.WebControls.FormDigest.RenderCore(SPWeb web) at Microsoft.SharePoint.WebControls.FormDigest.get_DigestValue() at Microsoft.SharePoint.SoapServer.SitesImpl.GetUpdatedFormDigestInformation(String url) at Microsoft.SharePoint.SoapServer.Sites.GetUpdatedFormDigestInformation(String url) at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at System.Web.Services.Protocols.LogicalMethodInfo.Invoke(Object target, Object[] values) at System.Web.Services.Protocols.WebServiceHandler.Invoke() at System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest() at System.Web.Services.Protocols.SyncSessionlessHandler.ProcessRequest(HttpContext context) at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) at System.Web.HttpApplication.PipelineStepManager.ResumeSteps(Exception error) at System.Web.HttpApplication.BeginProcessRequestNotification(HttpContext context, AsyncCallback cb) at System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags) at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags) at System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr pHandler, RequestNotificationStatus& notificationStatus) at System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr pHandler, RequestNotificationStatus& notificationStatus) at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags) at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags) 1195719c-0516-2045-a82e-8077f54daf22

02/06/2014 06:20:18.77 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation General ftd0 Medium Access Denied. Exception: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)), StackTrace: at Microsoft.SharePoint.Library.SPRequestInternalClass.RenderFormDigest(String bstrUrl, ISPDataCallback pFormCallback) at Microsoft.SharePoint.Library.SPRequest.RenderFormDigest(String bstrUrl, ISPDataCallback pFormCallback). 1195719c-0516-2045-a82e-8077f54daf22

02/06/2014 06:20:18.77 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation Claims Authentication af32v Verbose Claims Windows Sign-In: Sending 401 for request '

https://team.contoso.com/\_vti\_bin/sites.asmx' because the request is not from a browser. 1195719c-0516-2045-a82e-8077f54daf22

02/06/2014 06:20:18.77 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation General avfx Medium SOAP exception: System.Threading.ThreadAbortException: Thread was being aborted. at System.Threading.Thread.AbortInternal() at System.Threading.Thread.Abort(Object stateInfo) at System.Web.HttpResponse.AbortCurrentThread() at Microsoft.SharePoint.Utilities.SPUtilityInternal.SendResponse(HttpContext context, Int32 code, String strBody, String strContentType) at Microsoft.SharePoint.Utilities.SPUtility.IfNecessarySend401(HttpContext context, SPIisSettings iisSettings, Boolean isSOAP, Boolean isBrowserRequest) at Microsoft.SharePoint.Utilities.SPUtility.HandleAccessDenied(HttpContext context) at Microsoft.SharePoint.Utilities.SPUtility.HandleAccessDenied(Exception ex) at Microsoft.SharePoint.Library.SPRequest.RenderFormDigest(String bstrUrl, ISPDataCallback pFormCallback) at Microsoft.SharePoint.WebControls.FormDigest.RenderCore(SPWeb web) at Microsoft.SharePoint.WebControls.FormDigest.get_DigestValue() at Microsoft.SharePoint.SoapServer.SitesImpl.GetUpdatedFormDigestInformation(String url) at Microsoft.SharePoint.SoapServer.Sites.GetUpdatedFormDigestInformation(String url) 1195719c-0516-2045-a82e-8077f54daf22

If you scroll a bit above these entries, you can see that the actual user context is not the logged in user, but IUSR (who’s UserPrincipalName=i:0).w|s-1-5-17).

02/06/2014 06:20:18.49 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation Application Authentication ajezx Verbose SPApplicationAuthenticationModule: Authorization header doesn't contain Bearer, can't try to perform application authentication. 1195719c-0516-2045-a82e-8077f54daf22

02/06/2014 06:20:18.49 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation Authentication Authorization agb9s Medium Non-OAuth request. IsAuthenticated=True, UserIdentityName=0#.w|nt authority\iusr, ClaimsCount=17 1195719c-0516-2045-a82e-8077f54daf22

02/06/2014 06:20:18.49 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation Logging Correlation Data xmnv Medium User=0#.w|nt authority\iusr 1195719c-0516-2045-a82e-8077f54daf22

02/06/2014 06:20:18.72 w3wp.exe (0x1EF0) 0x1AB8 SharePoint Foundation General fbv6 Medium <?xml version="1.0" encoding="utf-8"?> <soap:Envelope xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="https://www.w3.org/2001/XMLSchema" xmlns:soap="https://schemas.xmlsoap.org/soap/envelope/"> <soap:Body> <GetUpdatedFormDigestInformation xmlns="https://schemas.microsoft.com/sharepoint/soap/" /> </soap:Body> </soap:Envelope> 1195719c-0516-2045-a82e-8077f54daf22

So, it seems like the request from CSOM is made as anonymous user rather than the logged in user.

Resolution:

Turns out this is an expected behavior and not really something to do with SharePoint Server 2013.  It’s documented here: https://support.microsoft.com/kb/979917.  Thankfully, ASP.NET make an application settings available to fix this issue in SharePoint Server 2013 as documented here: https://support.microsoft.com/kb/2686411.

So the fix is to ensure the following <appSettings/> key has the value false in it.

     <appSettings>
        <add key="aspnet:AllowAnonymousImpersonation" value="false" />
    </appSettings>

Look out for the below warning from the KB article.

There’s another way to fix this problem, which is to change <identity impersonate=”true” /> to <identity impersonate=”false” />.  But that will be unsupported from SharePoint perspective as per: https://support.microsoft.com/kb/2564833.

Hope this post was helpful!  This is Sridhar signing out until my next post!