Recently I was working with my colleague Lalitesh on a very interesting issue.
An ASP.NET MVC3 web application is deployed on 4 Windows Azure IAAS (equally applicable to PAAS Cloud Service as well ) Virtual Machines say machineA, machineB, machineC and machined. All these are in connected through a load balancer set HTTP-80. The ASP.NET MVC3 web application uses Forms Authentication.
1. During the first login into the application the request goes to say machineA. The session id is generated and given to the user logged in.
2. While the user is using the application suddenly she is redirected to the Login page.
3. Other users will also get kicked back to the logon page very frequently and randomly.
We verified that the session for all the instances is maintained in SQL Server on a separate Windows Azure IAAS VM.
<sessionState mode=”SQLServer” sqlConnectionString=”Server=Server_URL;Database=Database_Name;User ID=Login_ID;Password=Password;Trusted_Connection=False;Encrypt=True; Connection
Timeout=30;”cookieless=”false” timeout=”150″ allowCustomSqlDatabase=”true”/>
We looked at the login code to understand that it is using any session related information, at least to validate the user login. But it doesn’t. Then we Looked at the web.config file to confirm that the application is using Forms Authentication. We found an event in the Application event log from ASP.NET saying “Forms authentication failed – ticket supplied has expired”.
Event code: 4005
Event message: Forms authentication failed for the request. Reason: The ticket supplied has expired.
Event time: 10/10/2013 3:35:22 PM
Event time (UTC): 10/10/2013 3:35:22 PM
Event ID: dt9f30gc93014b0ca7ddu01471618g34
Event sequence: 3392
Event occurrence: 1
Event detail code: 50202
So we looked at the machineKeys and found that the machineKeys are not synced up between these IIS servers running on different Windows Azure IAAS VMs. To enable Forms Authentication ticket supplied from one server to be valid by another server if the request goes to it the machineKeys need to be in Sync. There are no sticky sessions in Windows Azure load balancer and the requests are directed in a round robin fashion.
To solve this issue we generated new machinekeys from the IIS manager as below , and pasted the <system.web/machineKey> element to the other server.
Go to the management console and then select the web site. Open the machine key configuration:
The machine key control panel has the following settings:
Here are the helpful articles that explains more about Forms Authentication and which helped us resolve the issue.
Understanding the Forms Authentication Ticket and Cookie
Explained: Forms Authentication in ASP.NET 2.0 – Web Farm Scenarios
Configuring Machine Keys in IIS 7
Lalitesh Kumar , Escalation Services, Windows Azure Technical Support
Angshuman Nayak, Escalation Services, Cloud Integration Engineering