500 and Other Errors in Azure Deployments

So you’ve been working feverishly on your Windows Azure application, all your testing and debugging in the Windows Azure development fabric (aka Windows Azure Emulator) is complete, life is good!  You deploy your application to your Windows Azure account in the cloud and wait in eager anticipation for the deployment to complete.   Browse to https://myreallycoolapp.cloudapp.net and ….

What can be more anticlimactic?  And where do you go next?

While working on the Bot Lab portion of the RockPaperAzure Challenge code, I ran into a few unexpected bumps in the road myself, a couple not even reproducible in my own environment in the cloud.   In all of those cases, everything worked just fine and dandy in the Windows Azure Emulator on my local machine (which should always be your first test anyway).

Here are three of the ‘bumps’ analyzed for you, in hopes that this post helps you avoid similar frustrations and time sinks!

Unhealthy Roles

In this case, my Web Role couldn’t even start up in Windows Azure – remember though, it worked just fine in the Windows Azure emulator.

There’s not a lot you can do at this point, unless you’ve thought ahead and set up remote desktop protocol (RDP) access; and yes, you can remote in even if your role isn’t in the Ready state.  Since I hadn’t set up my deployment for remote access, I ended up redeploying with the appropriate settings, but you could also contact Azure support for assistance at this point.

Because Bot Lab is deployed using Hostable Web Core (HWC) instead of full IIS, I couldn’t use IIS Manager to get more insight.  Rather, I needed to navigate to the diagnostics directory on the VM:

As you can see though, I didn’t have access to the LogFiles directory, but that was remedied by walking through the following prompts which appear after clicking Continue on the dialog above.  Essentially, I needed to grant privileges to the ad hoc user that is created to allow access to the VM via RDP.

Upon opening the log file, which is in W3C Extended Log File Format, the following line provided some insight:

2011-04-10 05:09:10 W3SVC1 RD00155D36031A 10.211.148.129 GET /do.__rd_runtime_init__ - 80 - 10.211.148.129 HTTP/1.1 WaWebHost-Agent - - 10.211.148.129 500 0 0 8415 158 6906

Highlighted above is the private server IP address and the HTTP status code, 500.   Browsing to the internal IP directly within the VM brought the root cause to light:

Had I deployed this site using IIS (which is the default with the Windows Azure SDK 1.3 and later), I would have seen this error right away in the browser on the client, and the Web Role would have actually been in the Ready state. 

So what’s the fix?   This error is the hallmark of a missing assembly, often resulting from a configuration error in which Copy Local has not been set when publishing from Visual Studio.  Indeed that was the case for some users who had downloaded the original source package.  My view of the source did and continues to show the following though:

Digging a little further, I found that the Copy Local property is stored in the .csproj file as the <private> tag… well, sometimes.  Both configurations below expose themselves as Copy Local True for me; however, the former configuration seems to present itself as Copy Local set to false for some.  If you flip Copy Local to false and then back to true, then the explicit <Private> tag is included and all is well.

Bottom line, Copy Local looks like a boolean value in the IDE, but it is really a ternary value under the covers.   Without an explicit <private> tag Visual Studio is supposed to do the right thing, but it seems that’s not the case for all.  You can read more about this behavior in Jeremy Jameson’s blog and Infovark.

Web.config Misconfiguration

This one presented itself a bit differently: the Windows Azure portal showed the Web Role in a Ready state, but browsing to it yielded the dreaded 500 error:

Using the same methodology as mentioned above, I used remote desktop to check out what’s happening on the VM.  Browsing to the page via the internal IP address resulted in:

Here, the <system.web.extensions /> tag was the culprit.   System.web.extensions defines behavior for ASP.NET Ajax pages, and the element in the web.config was part of the original open source download that we leveraged for the Rock Paper Azure project.  The extension isn’t really needed given the modifications we made (to incorporate jQuery), but  it remained in the web.config.  In the end, I just removed the tag, but the real resolution and explanation for the behavior can be found on Avkash Chauan’s blog.

Blank Startup Page

Bot Lab is an ASP.NET MVC 2 application and so uses routing to resolve URLs.  When you first install and browse to the site, the initial URL is https://localhost:<port>/InitialSetup, which should result in execution of the Index method in IntialSetupController.cs and display an initial login screen.  In some cases, it was just showing a blank screen.

When pages don’t display correctly in an MVC application, the usual culprit is a misrouted request (recall for an MVC application, you explicitly provide Route definitions, typically in the Application_Start event of global.asax).  An incorrect routing setup seemed unlikely here, since the same code seemed to work for many others, but just in case, I leveraged Phil Haack’s ASP.NET MVC Route Debugger.  That confirmed the correct route was being selected, but it was still failing.  Additionally, a brand new basic MVC project was also working fine, just not Bot Lab!

It turns out the project working fine as a standalone application was a bit of a red herring, since it uses the development web server (Cassini) by default.  Had I tested with full IIS, the MVC site wouldn’t have worked either, Azure notwithstanding.

The culprit: HTTP Redirection was not enabled.   Simply selecting the option via the Turn Windows features on off dialog, accessible from Control Panel>Programs resolved the MVC failures (see right). 

I also noticed that Windows Communication Foundation HTTP Activation was not selected, despite being required per the Azure SDK installation documentation.