"Failed to verify user permissions" error when using DspSts.asmx web service

I recently handled a case where the customer sees a "Failed to verify user permissions" error when accessing the DspSts.asmx web service in SharePoint 2007. Below is the code similar to what they were using to invoke the Query method of DspSts.asmx web service. It's also very similar to the same provided in SharePoint SDK. Surprisingly, they saw this error only in their environment and we were unable to replicate it at our end.

The code:

    1:             string siteUrl = ConfigurationSettings.AppSettings["siteUrl"].ToString();

    2:             string username = ConfigurationSettings.AppSettings["username"].ToString();

    3:             string password = ConfigurationSettings.AppSettings["password"].ToString();

    4:             string domain = ConfigurationSettings.AppSettings["domain"].ToString();

    5:             dsptest.StsAdapter stsAdapter = new dsptest.StsAdapter();

    6:             stsAdapter.Url = siteUrl + "/_vti_bin/DspSts.asmx";

    7:             stsAdapter.Credentials = new NetworkCredential(username, password, domain);

    8:             string selectedList = comboBox1.SelectedItem.ToString();

    9:             string selectedListGuid = selectedList.Substring(selectedList.IndexOf("|")+2);

   10:             string[] vArray = new string[1];

   11:             vArray[0] = "1.0";

   12:             dsptest.Versions versions = new dsptest.Versions();

   13:             versions.version = vArray;

   14:             stsAdapter.versions = versions;

   15:             dsptest.RequestHeader reqHeader = new dsptest.RequestHeader();

   16:             reqHeader.document = dsptest.DocumentType.content;

   17:             reqHeader.method = dsptest.MethodType.query;

   18:             stsAdapter.request = reqHeader;

   19:             dsptest.QueryRequest myRequest = new dsptest.QueryRequest();

   20:             dsptest.DSQuery sQuery = new dsptest.DSQuery();

   21:             sQuery.select = "/list[@id='" + selectedListGuid + "']";

   22:             myRequest.dsQuery = sQuery;

   23:             dsptest.DspQuery spquery = new dsptest.DspQuery();

   24:             myRequest.dsQuery.Query = spquery;

   25:             try

   26:             {

   27:                 XmlNode xmlnode = stsAdapter.Query(myRequest);

   28:                 textBox1.Text = xmlnode.OuterXml;

   29:             }

   30:             catch (Exception _e)

   31:             {

   32:                 textBox1.Text = "Error: " + _e.Message + System.Environment.NewLine +

   33:                     "Stack Trace: " + System.Environment.NewLine +

   34:                     _e.StackTrace;

   35:             }

On investigating this issue, we found that they have enabled anonymous access at their web application level.  The problem is they had enabled anonymous access ONLY at their web application level.

The way to enable anonymous access at web application level is:

Browse to central administration site > application management > authentication providers (under application security)

Select the default provider where you wish to enable anonymous access (shown below)

And enabled anonymous access

When we do this, the web site at the IIS level will have its anonymous access enabled when we check the Directory Security tab from IIS MMC.  However, site collections within this web application are not yet configured to work with anonymous access.  Since the DspSts.asmx web service sits within the site collection, it looks like it has some problem understanding the call that's made from an anonymous contexts and it fails with the error:

    1:    Error: Failed to verify user permissions.

    2:    Stack Trace: 

    3:    at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)

    4:    at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)

    5:    at PPSIssueTester.dsptest.StsAdapter.Query(QueryRequest queryRequest) in C:\Documents and Settings\Administrator\Desktop\PPSIssueTester\PPSIssueTester\Web References\dsptest\Reference.cs:line 127

    6:    at PPSIssueTester.Form1.button2_Click(Object sender, EventArgs e) in C:\Documents and Settings\Administrator\Desktop\PPSIssueTester\PPSIssueTester\Form1.cs:line 110

These are the scenarios we've been testing:

1. NTLM authentication setup at both web application and site collections level with anonymous access enabled no where - call to DspSts.asmx works.

2. Anonymous access enabled at both web application and site collections level - call to DspSts.asmx works.

3. Anonymous access enabled only at web application and not at site collections - call to DspSts.asmx fails with the above error.

Well, the easiest resolution is to either enable anonymous access at site collections as well or to simply use NTML authentication (or form authentication) without enabling anonymous access.  It's another question as to whether having authentication configured this way is recommended or not (i.e., enabling anonymous access only at the web application level).  None of the SharePoint components fail by having authentication configured this way - per my understanding.  But apparently, this configuration seems to have some problem in terms of using DspSts.asmx web service.