Custom Probe + Port ACLs on ASP.NET Web Farm

  • Article

I was asked recently how to take a Windows Azure Infrastructure Services VM (IaaS VM) out of the load balancer rotation so it could be updated.  This particular partner is using IaaS VM’s as a stepping stone on the way to a PaaS service and so needs to set up a fairly typical ASP.NET web farm.  In his set up on premises he can easily remove a machine from the load balancer during updates and he needs to replicate this capability in the cloud.  I didn’t actually know the answer to this, so checking my internal network I discovered that a custom load balancer probe would do the trick.  This blog post examines the technique in detail.

I will give an overview of steps along the path to these goals:

  1. A farm of 3 VMs running a simple ASP.NET application, load balanced on port 80
  2. A custom probe defined on port 80 that
    • is easy to switch the state of
    • is easy to verify
  3. A secure deployment mechanism.  When playing around with this, it became obvious that a simple, fast and secure deployment mechanism is a real handy tool to have.

This blog post was written in June of 2013.  I’m using Visual Studio 2012 and the 2.0 version of the Windows Azure SDK.  I also make use of the June 2013 update to the Windows Azure PowerShell Cmdlets.  Here’s a link to the downloads page where both of these items can be acquired.

This post is a variation on Michael Washam’s blog: Publishing and Synchronizing Web Farms using Windows Azure Virtual Machines
(Thanks, Mike!)

And of course, the core MSDN documentation: Load Balancing Virtual Machines

The steps that I followed to set up this demonstration:

  1. Create a Web Server VM image
    1. Windows Server 2012 gallery image from the Azure portal
    2. Add Application Server Role
    3. Add Web Server Role
    4. Add ASP.NET 4.5 feature of Web Server Role
  2. Set up a 3 VM farm based on this image
    1. Add port 80 endpoint, load balanced, custom probe defined
    2. Add port 8080 endpoint for web deploy operations
  3. Install and configure Web Deploy 3.0 on each instance
    1. I set it up on port 8080 on each instance
    2. I used different public ports for each, so Visual Studio could target each one
  4. Install a simple ASP.NET web app on each instance
    1. This app has 2 pages: Default.ASPX and CustomProbe.ASPX
    2. CustomProbe.ASPX has a test in it that allows you to switch the load balancer state
  5. Set up an endpoint ACL for the web deploy port.

 

Step 1/2/3:

These steps are pretty close to those detailed in Michael Washam’s blog above.  The only distinctions I would make are:

  1. To run an ASP.NET 4.5 app I needed to install the ASP.NET 4.5 feature of the Web Server Role.
  2. I installed the 8080 web deploy port on all 3 VMs.  I used different public ports on each, so 8080, 8081, 8082 (public) and 8080 (private).  This is so I could target each instance individually with Web Deploy from Visual Studio.  Remember – the reason for taking this approach is that the partner wants to update one instance at a time, and leave the other instances running. 
  3. I set up Web Deploy exactly the same internally, including the firewall adjustment.
  4. I did not set up the Web Deploy Sync capability that Mike does in his post.
  5. In the set up of port 80, ProbePath is ‘/CustomProbe.ASPX’

Here’s my Powershell script:

image

Step 4:

I used an empty ASP.NET application, added Default.ASPX and CustomProbe.ASPX. 

I added just enough content to Default.ASPX so I know when it’s loaded. 

I added a simple line of code to CustomProbe.ASPX to implement the load balancer state switch:

 

image

In addition to this, I tweaked IIS slightly – I turned off “Default Web Site” and created a new site “TestWebStatusCodes” into which I installed my little ASP.NET app.

Step 5:

The Web Deploy port is secure by obscurity (not on a default port setting), but we now have the ability to go one step further.  As of June 3, we now have the ability to assign ACLs to public endpoints.

Here’s a blog post with details:  Windows Azure PowerShell June 2013 Update for IaaS and PaaS

And here’s the snip that I used to protect my web deploy port:

image

TIP: In the Get-AzureVM call, even though it seems that you’re uniquely identifying a VM by -Name (iis1 above), if you don’t specify the –ServiceName, you get back the wrong kind of object.

Here’s a snip from my IIS logs, showing where I switched the load balancer state for one VM:

image

CustomProbe.ASPX looks for a file called “TakeMeOffline.TXT”.  If it finds it, it returns a 410.  This takes the VM out of rotation.  When I’m done maintaining the machine, I remove or rename the file.  (“DontTakeMeOffline.TXT”)

And that’s it!  Cheers!

Credits:

Thanks to Neil Mackenzie for being part of that internal network earlier mentioned.  And of course, Michael Washam and his PowerShell genius.