[Azure Batch] Server failed to authenticate the request

  • Article

Today happen to work on this problem, where developer running unit test code creating Azure Batch Jobs in row and check the status in a tight loop getting forbidden error. Interestingly, it worked for him for the first couple of calls “job” creation but fails continuously after that in row. Grabbed the the fiddler log to see the request and response and noticed something as like below. 

Fiddler trace:

HTTP/1.1 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

Content-Length: 864

Content-Type: application/json;odata=minimalmetadata

Server: Microsoft-HTTPAPI/2.0

request-id: fd81d621-xxxxx-b252-1db46324f1a6

Strict-Transport-Security: max-age=31536000; includeSubDomains

X-Content-Type-Options: nosniff

DataServiceVersion: 3.0

Date: Tue, 13 Dec 2016 15:51:34 GMT

Cache-Control: proxy-revalidate

Proxy-Connection: Keep-Alive

Connection: Keep-Alive

{

  "odata.metadata":"https://xxxxxx.eastus2.batch.azure.com/$metadata#Microsoft.Azure.Batch.Protocol.Entities.Container.errors/\@Element","code":"AuthenticationFailed","message":{

    "lang":"en-US","value":"Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:xxxxx-564c-4930-xxx-xxx\nTime:2016-12-13T15:51:34.2917937Z"

  },"values":[

    {

      "key":"AuthenticationErrorDetail","value":"The MAC signature found in the HTTP request '6nxk/xxxxxx/xmdrWt55RnMRmsg=' is not the same as any computed signature. Server used following string to sign: 'GET\n\n\n\n\n\n\nTue, 13 Dec 2016 15:42:18 GMT\n\n0x8D4236E9EB80606\n\n\nocp-date:Tue, 13 Dec 2016 15:51:35 GMT\n/xxxxx/jobs/deleteJob0\napi-version:2016-07-01.3.1'."

    }

  ]

}

Findings:

This appears to be some kind of caching - https://social.msdn.microsoft.com/Forums/SqlServer/en-US/67183d62-60ab-4ef0-a1ca-b765d85ea2f6/authenticationfailed?forum=azurebatch.  It is caused by client-side caching at the proxy layer. Usually the proxy server at client side caches the get responses and for subsequent gets proxy server will try to serve the request from its cache. It sends a different request to server by adding If-Match header with the ETag. However, this causes a problem in this case because the proxy server does not change the Authorization header, it uses the same Auth header that client sends. Hence, the auth header that client sends does not match what server expects.

Fix:  

Added the below config value to App.Config file which resolved this issue. He was able to continue execute his test harness by submitting/deleting 10 jobs successfully without caching issues.

<system.net>

    <requestCaching defaultPolicyLevel="NoCacheNoStore"/>

Hope this helps.