Applying a “Web and Worker Roles” pattern to the Azure App Service

Summary: A common use for Azure Cloud Services is to implement services that utilise a “web and worker role” pattern to offload complex processing to async workers. This article describes how you can implement the same pattern with the Azure App Service. This article is from Tom Hollander’s blog.

Azure Cloud Services was one of the first technologies available in Microsoft Azure, and it has served us well for many years. And while there are still some workloads that are well-suited for Cloud Services today, I’m finding that these days I’m using and recommending the Azure App Service (the new technology built on the foundation of the older Azure Websites) more and more. Some of the key benefits of the App Service include built-in support for multiple development platforms, near-instant deployment, ability to deploy using Git and FTP, simplified management, an extensibility model, and support for Mobile Apps, Logic Apps and API Apps in addition to Web Apps.

The Azure App Service is a great container for a wide range of application types, but often I see people overlook this option because they want to use the “web and worker role” pattern made famous by Cloud Services. This pattern is shown in the following diagram and involves:

• A Web Role (consisting of 1 or more instances) that hosts a website or an API and is accessed by the application’s users. The web role can perform simple tasks synchronously, but whenever any complex processing is required it will create a message and drop it on a Queue.
• One or more Queues, which support asynchronous communication between the Web Role and the Worker Role.
• A Worker Role (consisting of 1 or more instances) which pull messages of the Queue and perform slow or complex processing tasks.
• Storage (such as Tables, Blobs or SQL Database) that stores the system’s state and typically includes the result of the Worker Role’s processing tasks.

This is a great pattern for many applications as it keeps the UI or API in the Web Role responsive, it allows the Web Role and Worker Role to scale independently, and it provides resilience in case the Worker Role instances are bogged down or unresponsive. This article explains how you can implement this pattern easily using Web Apps in the Azure App Service. At the end of the article I’ll provide a few examples of situations where you may still want to use Cloud Services instead.

Note: Azure’s capabilities and APIs change rapidly. This information is current as of April 2015. If you’re reading this in the distant future, you may want to check with other sources on current capabilities and best practices.

Using WebJobs for Async Processing

As we look to implement this pattern with Web Apps on the Azure App Service, the obvious starting point is the Web Role. A Web App provides an IIS-based container for deploying web sites, so it’s a natural fit to deploy our front-end code.

For the Worker Role, we need to look to WebJobs. WebJobs are non-web applications that can run within the context of an Web App, either continuously or on a schedule, and are ideal for background processing tasks. You can implement WebJobs as arbitrary executables or scripts, but for new development using .NET I thoroughly recommend the WebJobs SDK. The WebJobs SDK provides a declarative and event-driven programming model that makes it easy to read messages from queues and write to tables and storage, as well as handling management tasks such as logging and poison message handling.

For example, I built a WebJob using the WebJobs SDK to calculate the prime factors of a number (always an important capability in any solution!). Note the use of the attributes in the method signature that specify that the method should be automatically invoked when a message arrives on the factorize-in queue, and that the results will be added to a table called primefactors.

 public static void ProcessQueueMessage([QueueTrigger("factorize-in")] string number, 
    [Table("primefactors")] ICollector<FactorsEntity> tableCollector, TextWriter log)
{
    log.WriteLine("Starting to factorize " + number);
    DateTime start = DateTime.Now;
    long[] factors = Integers.FactorizeFast(number);
    DateTime stop = DateTime.Now;
    var factorsEntity = new FactorsEntity
    {
        Factors = String.Join(",", factors
                        .Select(s => s.ToString())
                        .ToArray()),
        RowKey = number,
        PartitionKey = DateTime.Now.ToString("yyyy-MM-dd"),
        CalculationTime = (stop - start).TotalMilliseconds,
    };
    tableCollector.Add(factorsEntity);

    log.WriteLine("...and the factors are: " + factorsEntity.Factors);
}

Using App Service Plans to manage your topology

From a functional perspective, the web site and the WebJob are all we need to implement the pattern. However another key attribute of the pattern is its physical topology, with the Web Role and Worker Role independently scalable and running on different sets of VMs. A WebJob meanwhile runs in the context of a Web App and shares its resources. For simple sites sometimes this is exactly what you want, but for more complex sites with significant backend processing work it can be important to provide dedicated resources for each of the roles.

Fortunately you have complete control over how your Web Apps are hosted within the Azure App Service. First, it’s important to remember that a Web App is a logical concept – it includes the code and content for your web site, any WebJobs, and some configuration metadata, but it doesn’t directly map onto anything tangible like a VM.

The thing that makes your Web App run somewhere is called an App Service Plan (formerly known as a Web Hosting Plan or a Server Farm). An App Service Plan refers to the set of infrastructure in which your Web Apps run. If you choose a Free or Shared plan, your Web App runs on a VM alongside other Web Apps from other customers, but if you choose a Basic, Standard or Premium plan then you get one or more VMs to yourself. You can also choose to run multiple Web Apps within the same App Service Plan. The following diagram shows one App Service Plan with 3 VMs, each running the same 2 Web Apps.

When you look at it this way, an App Service Plan is starting to look a little like a Role from the Cloud Service world. So to implement our “Web and Worker Role” pattern in the App Service, we can simply add another App Service Plan to represent the second role. Note that there’s no concept of “Web” versus “Worker” App Service Plans, but there’s nothing stopping us from deploying only web code to the Web App in one plan, and only WebJobs to the Web App in the other plan. So our solution looks like the following, with 2 App Service Plans, each with their own Web App, one which contains only web code, and one which contains only WebJobs.

Pros and Cons for App Service versus Cloud Services

Now that we’ve successfully implemented the “Web and Worker Role” pattern using Web Apps on the Azure App Service, let’s take a step back and compare the results.

Both solutions share the same VM topology that we were looking for, and both allow each “role” to be scaled independently.

One key difference is that a Cloud Service is always deployed as a unit, ensuring that all roles remain in sync, but at the cost of deployments being slower and potentially more disruptive. With our App Service solution, you can deploy each Web App individually which is faster but potentially riskier if you deploy incompatible code to each. Still it’s worth noting that with the App Service solution you can still put everything into the same Resource Group which can help you deploy and manage the solution as a single unit.

You may still need to use Cloud Services if your solution requires custom software to be installed on the VM, if you need to run code with elevated privileges, or if you need to be able to remote into the VMs. Also, Cloud Services provide a wider range of VM sizes to choose from, and you can scale beyond the maximum 20 instances allowed in (Premium) App Service Plan. 

However App Services provide a range of additional capabilities, including multiple deployment slots, Testing in Production, Git, WebDeploy and FTP deployments, built-in support for Java, PHP and Node, and much more, which is why it’s currently my favourite way to deploy applications on Azure.