Scaling cloud apps with the .NET Framework 4.5

Brandon Bray

The .NET Framework 4.5 is now available on Windows Azure. Thank you to everyone cheering for this moment. We’re just starting to see the possibilities that a rich developer framework like the .NET Framework and the cloud can have together. Richard Lander, a program manager for the Common Language Runtime, explains a few best practices that can result in fantastic scaling by applying very simple coding patterns around the .NET Framework 4.5 asynchrony feature. –Brandon

The .NET Framework 4.5 is now available on Windows Azure

At the Build 2012 event, the Windows Azure team announced that you can use the .NET Framework 4.5 to build and deploy apps for Windows Azure. You can use many of the features that we made available in the .NET Framework 4.5, including C# 5 and Visual Basic 11 features that will help increase your productivity and make your apps faster and able to scale in the cloud.

Please download the new update to the Windows Azure SDK. The new SDK is available for both Visual Studio 2010 and 2012. For .NET Framework 4.5 development, you’ll need to use Visual Studio 2012 or Visual Studio 2012 Express for Web.

We have enabled the .NET Framework 4.5 in the following Windows Azure services:

• Windows Azure Cloud Services
• Windows Azure Web Sites
• Windows Azure Virtual Machines

By default, Cloud Services apps are now deployed on Windows Server 2012, which includes the .NET Framework 4.5. Windows Azure Web Sites are deployed on Windows Server 2008 R2 machines, which also include the .NET Framework 4.5. They will soon be deployed to Windows Server 2012 machines, like Cloud Services apps. In addition, you can opt to use Windows Server 2012 images when you deploy Virtual Machines. I hope that you enjoy using the .NET Framework 4.5 with these Windows Azure services.

You can see the new Windows Azure admin portal below for Windows Azure Web Sites. The Configure tab enables you to select the .NET Framework version to use for your site. The .NET Framework 4.5 is selected by default, as opposed to the .NET Framework 3.5. Both .NET Framework 4 and 4.5 sites run on the .NET Framework 4.5.

There are many reasons to start using the .NET Framework 4.5 in Windows Azure. We’ve talked about some of them, including the new background server GC, on this blog. There are also many new features within ASP.NET and Windows Communication Foundation (WCF), such as HTML5 WebSockets. The must-have feature in the .NET Framework 4.5 is async, which lends itself perfectly to achieving scale in the cloud. Let’s take a look.

.NET Framework async is a cloud-scale programming model

When we started working on .NET Framework 4.5, we wanted to enable .NET Framework apps to scale much better, particularly in a cloud environment like Windows Azure. The .NET Framework already has several features that enable parallelism through threads. Those features—for instance, the .NET Framework thread pool, the BackgroundWorker class, and others—are great for reducing the latency of requests to your website or service. On the Windows platform, threads alone are not enough to enable maximum scaling. To get the most out of your app, you need to also make use of asynchronous I/O built into the operating system. For years, this has been very hard to do, but now with async in the .NET Framework, it has become very simple.

We’ve been talking about the new task-based asynchronous programming model this year on the .NET Framework blog. Please take a look at our earlier posts to better understand the basics of async in the .NET Framework. Now that Windows Azure supports the .NET Framework 4.5, it’s time to take a look at how async can scale your apps, and how it makes cloud programming with the .NET Framework better.

For the cloud, the primary goal is to serve up website or service requests as quickly as possible. Another very important goal is to limit the number of machines or VMs that you need to make available to keep up with demand. Asynchrony is very effective at helping you achieve this second goal of scaling and make better use of your hardware.

I’m sure that you’ve written ASP.NET or WCF code that parallelizes I/O calls to the file system, network, or database by spinning off multiple threads and performing each I/O call on a separate thread. In those cases, many of your threads spend time sitting idle, waiting for some other system (like the file system) to return some data. If your server gets many requests at the same time, then the number of threads (many of which are potentially waiting idle at any given point) starts to grow. Large numbers of idle threads can result in a significant tax on memory.

.NET Framework async approaches the challenge of scaling in a very different way. It returns a thread to the thread pool when it is waiting on an operation to complete that doesn’t require the CPU, such as file I/O or networking. This makes that thread available for other operations. As a result, you can expect that async will use fewer threads at a time, requiring less memory, which enables your site to scale.

Async programming with the .NET Framework is not new. You’ve been able to program with the IAsyncResult interface, for example, for years now. The big advance in the .NET Framework 4.5 is an async model that is much easier to use because it has great support in Visual Studio 2012, the managed languages, the .NET Framework class libraries, and the CLR. You can now take advantage of this model in the code that you deploy to Windows Azure, to scale your sites and services.

Async enables your web server to scale

Async is a great general model for the cloud, and is a major win for scaling in ASP.NET in particular. ASP.NET requests threads from the CLR thread pool to serve incoming requests. When demand is high and ASP.NET requests a large number of threads from the thread pool, the latency of requests and memory usage may start to climb. For example, if your Web Form event or your MVC action method is blocked on I/O, ASP.NET may need an increasing number of threads to service requests. The more threads you use, the more memory you will use, which can eventually lead to disk-backed memory paging, which can significantly affect response time. Instead, you can use async to use the threads available to your web server much more efficiently.

Making an event in a Web Form or an action method in MVC asynchronous enables ASP.NET to use threads much more efficiently. The async model enables you to serve more requests on the same hardware. Your job as a web programmer is to identify the scenarios where your code would benefit from asynchrony, by awaiting async calls and enabling ASP.NET to scale. As a general rule of thumb, if the .NET platform has gone to the effort of providing an async API for a given scenario, that is probably a scenario that will benefit from asynchrony.

Async support in ASP.NET

Async support has been added to ASP.NET 4.5, enabling you to write task-based async code within ASP.NET pages, http handlers, and http modules. This support includes both ASP.NET Web Forms and MVC.

All APIs that use the Task-based Async Programming model (TAP) need to adhere to certain requirements. For instance, all async methods should return Task or Task<T> objects. Let’s take a look at HttpTaskAsyncHandler. This is the new base type for HTTP handlers that are built with the new async model. You will need to use this type if you want to call async APIs within HTTP handlers. The following example demonstrates a very minimal async HTTP handler implementation. Since the ProcessRequestAsync method is marked as async and returns a Task, you are free to await async calls in your HTTP handler implementation, as you see below.

public class MyAsyncHandler : HttpTaskAsyncHandler
{
    // ASP.NET automatically takes care of integrating the Task-based override
    // with the ASP.NET pipeline.
    public override async Task ProcessRequestAsync(HttpContext context)
    {
        HttpClient client = new HttpClient();
        String result = await client.GetStringAsync("http://msdn.com/netframework");
        // Do something with the result.
    }

}

Before – synchronous (blocking) version:

public ActionResult Index()
{
    WebClient client = new WebClient();
    var pageString = client.DownloadString("http://asp.net");
    ViewBag.PageString = pageString;

    return View();
}

public async Task<ActionResult> IndexAsync()
{
    var client = new WebClient();
    String pageString = await client.DownloadStringTaskAsync("http://asp.net");
    ViewBag.PageString = pageString;

    return View();
}

ASP.NET Web Forms has also been updated to enable you to write async code, as you will see in the next section.

Asynchronous code example in ASP.NET Web Forms

Let’s take a look at a concrete case of asynchronous code written as an ASP.NET Web Forms project. This app uses the .NET Framework networking APIs to call into REST APIs exposed by a music service. You’ll notice that the type and method names are music-oriented, which matches the service.

Notice that I use the async and await keywords in the body of the code, but then I compose tasks together at a higher level within the app by using the Task.WhenAll method. The advantage of this approach is that you can request that an aggregated set of operations be run at once, providing your site with the benefits of parallel behavior with the lower cost of an asynchronous implementation. This approach is particularly useful if the operations spend a disproportionate amount of time waiting for the network file system to return data.

Let’s take a look at the code. First, I’ll make sure that my project targets the .NET Framework 4.5; otherwise, I won’t get access to all the new features. Next, I have to enable asynchronous processing within my ASP.NET Web Forms app by using the Async page directive:

<%@ Page Async="true"

Similarly, I have to enable async code within the Page_Load event, by adding the async keyword before the return type:

protected async void Page_Load(object sender, EventArgs e)

Now that both of those are in place, I can write some async code. My example is a minimal implementation of a site that calls into a music service to get music track information. The GetTracksAsync method gets a list of REST APIs to call, each of which represents a separate artist, and then calls into the GetArtistTracksAsync method to get the tracks available for each artist. GetArtistTracks also contains JSON deserialization code. You can safely ignore that part of the example.

protected async void Page_Load(object sender, EventArgs e)
{

    List<Track> trackList= await GetTracksAsync();

    // Now do stuff with all of the music tracks.

}

async Task<List<Track>> GetTracksAsync()
{
    List<string> urls = GetRestURLs();
    var hype = new Hype();

    List<Track>[] results = await Task.WhenAll(
        from url in urls
        select hype.GetArtistTracksAsync(url));
    return results.SelectMany(r => r).ToList();
}

//Hype.GetArtistTracksAsync
public async Task<List<Track>> GetArtistTracksAsync(String uri)
{
    WebClient client = new WebClient();

    // This line is the important one.
    // Asynchronous call over the network.
    String json = await client.DownloadStringTaskAsync(uri);
    
    // JSON deserialization -- also async.
    Dictionary<String, JToken> jTokens = await JsonConvert.DeserializeObjectAsync<Dictionary<String, JToken>>(json);
    List<Track> tracks = new List<Track>();

    foreach (String key in jTokens.Keys)
    {
        if (key == "version")
            continue;

        Track track = jTokens[key].ToObject<Track>();
        tracks.Add(track);
    }

    return tracks;
}

I like this approach, since it enables a potentially parallel behavior efficiently. This is what Task.WhenAll gives you. What would happen if I’d simply used await everywhere, without Task.WhenAll? The await keyword behaves sequentially, since it needs to satisfy the procedural nature of your code. It is sequential, but it is not synchronous. Every time your code awaits an async call, you’ve provided an opportunity for the async system to return your thread to its caller, while you await that call. Note that the caller could be the CLR thread pool. And the code that follows an await will be executed only after the awaited entity has entirely completed. Some of your code may be able to take advantage of Task.WhenAll to enable parallelism; however, you should make extensive use of awaiting async APIs to enable scaling through asynchrony.

The following code is a commented non-linq expansion of GetTracksAsync, if you are not familiar with linq syntax. It provides an easier to read version of what the code above is doing.

async Task<List<Track>> GetTracksAsync()
        {
            List<String> urls = GetRestURLs();
            // 'tasks' is the list of tasks that the code collects in the for loop,
            // which can then be awaited as a group with Task.WhenAll.
            List<Task<List<Track>>> tasks = new List<Task<List<Track>>>();
            List<Track> trackList = new List<Track>();
            // Encapsulates Hype Machine service – http://hypem.com.
            Hype hype = new Hype();

            // Collect Tasks for all work to be done.
            // Notice the lack of the await keyword here.
            // We want Tasks!            
     foreach (String url in urls)
            {
                Task<List<Track>> task = hype.GetArtistTracksAsync(url);
                tasks.Add(task);
            }

            // This is the important line.
            // All work is collected as Tasks and then aggregated into a single task to
            // await with Task.WhenAll.
         // Task.WhenAll lets you to await all the Tasks in one statement 
         // allowing then to potentially run in parallel.
            await Task.WhenAll(tasks);

            // Aggregate results into a single List<Track> to return to caller.
            foreach (Task<List<Track>> task in tasks)
            {
                // We can await a task as often as we like. 
                // We know that all tasks have already completed. 
  // Awaiting an already completed task is a very lightweight operation that completes immediately.
  // Using the await here (rather than, for instance, task.Result) allows potential errors to propagate correctly.
                trackList.AddRange(await task);
            }

            return trackList;

        }

Async performance analysis

I did an unscientific performance analysis of three versions of the code sample above. First, I have the example above, which uses Task.WhenAll. Next, I created another version that always awaits async calls, with no use of Task.WhenAll. Last, I created a fully synchronous version that calls only synchronous APIs – not APIs that return Task. To make the analysis meaningful, I provided several artists to query for, resulting in a significant number of REST payloads to request and process. I conducted the experiment within an ASP.NET Web Forms app running under CTRL+F5 — retail build. Let’s take a look at the performance results.

The numbers really tell the story. The low number for the asynchronous – Task.WhenAll approach demonstrates the value of the parallel approach. My app returns quickly and is likely to scale well, because it is utilizing an asynchronous implementation from the top to the bottom of the call stack. The plain asynchronous – await model is also good, because it enables similar throughput as a synchronous model, but again, it can be scaled more efficiently with an asynchronous implementation. Last, the synchronous model example is the right option when an async option doesn’t exist. As you see, you now have several approaches to meet your site’s throughput and scaling needs.

Compute-bound operations are a good example of the synchronous model. They can benefit from parallelism (by using threads); however, because compute-bound operations do not sit idle, unnecessarily occupying a thread, there is no thread that the async infrastructure can return to the thread pool for use by other operations. Therefore the .NET Framework does not offer async versions of compute-bound operations directly. If you want to make sure that a compute-bound operation is executed on the thread pool and off your current thread, you can pass it to the Task.Run(..) method. This method returns a Task representing the completion of your operation, so you can await and compose its results in the same manner as other asynchronous operations.

Taking a look at how the code for Task.WhenAll and await differ

It is useful to contrast the two different asynchronous implementations. I found that there are minimal code differences between the two asynchronous approaches, at least in the way I used them in my sample. It comes down to how async calls are handled within the for loop in the example. Let’s take a look at the two cases.

Here’s the Task.WhenAll example again:

async Task<List<Track>> GetTracksAsync()
        {
            List<string> urls = GetRestURLs();
            var hype = new Hype();

            List<Track>[] results = await Task.WhenAll(
                from url in urls
                select hype.GetArtistTracksAsync(url));
            return results.SelectMany(r => r).ToList();
        }

async Task<List<Track>> GetTracksAsync()
{

    List<String> urls = GetRestURLs();
    List<Track> trackList = new List<Track>();
    Hype hype = new Hype();

    foreach (String url in urls)
    {
        List<Track> tracks = await hype.GetArtistTracksAsync(url);
        trackList.AddRange(tracks);
    }

    return trackList;
}

The two examples are very similar. The difference is that the second one awaits  hype.GetArtistTracksAsync after each call. This means that although the system can use some of the resources occupied by my program for other things while the program is waiting for the data to arrive over the network, the program flow can only continue after each set of tracks has been retrieved. Conversely, in the first version I request all required sets of tracks to be downloaded and then await them all together. This allows the system to share the resources between the download operations and perform them concurrently if there are enough system resources available. As a result, the downloads are likely to complete sooner.

An interesting observation is that the two versions of GetTaskAsync are able to satisfy the same method signature, specifically the return type. The way that one implementation aggregates tasks is purely an implementation detail. Another observation is that both methods are able to call and rely on the same async method, Hype.GetArtistTracksAsync. This method did not need any knowledge of the different approaches to awaiting the resulting tasks. Therein lies a big part of the value of the task-based model.

As you start to use async within your sites and services, please do spend some time looking at how these different models affect your performance, both in terms of throughput and memory.

Async support in WCF

Async support has also been added to WCF. The async benefits and model in WCF are very similar to ASP.NET, and they enable you to scale your WCF services much better on the server. We have also enabled automatic generation of async proxy methods for your service operation, which will enable client apps to easily call service operations asynchronously.

WCF services expose operation contracts for clients to implement and call. In WCF 4.5, the use of the asynchronous programming model on either the client or the service is a local implementation detail that is not reflected in the operation contract. The wire format of a message that a client sends to a service (or vice versa) remains the same, enabling the client and service to independently choose either synchronous or asynchronous implementations. What has changed is the local behavior. If the client calls a service operation asynchronously, the client will not block any threads while making the service call, enabling the client UI to remain responsive. On the service side, if the operation is implemented asynchronously, the async system can return the thread back to the CLR thread pool to enable better scaling of the service.

Writing async service code

As in ASP.NET, it is pretty easy to take advantage of async APIs within a WCF service. It is as simple as adding an async method to your service, which will enable you to await async APIs. As you can see below, I added a single async method, highlighted in yellow, within the code provided with the default WCF Service Application template in Visual Studio 2012. I am able to await WebClient.DownloadStringTaskAsync, to take advantage of the scaling benefits that come from an asynchronous approach. To enable your service to scale, you should choose an async implementation for I/O-intensive processing in your services.

public class Service1 : IService1
    {

        public string GetData(int value)
        {
            return string.Format("You entered: {0}", value);
        }

        public async Task<String> GetSomeDataAsync(int value)
        {
            WebClient client = new WebClient();
            return await client.DownloadStringTaskAsync("http://msdn.com/");
        }

    }

You’ll notice – and this should be no surprise – that the IService1 service contract exactly matches the implementation in Service1.

[ServiceContract]
public interface IService1
{

    [OperationContract]
    string GetData(int value);

    [OperationContract]
    Task<string> GetSomeDataAsync(int value);

}

Writing async apps that consume WCF end-points

It is very beneficial to write async code on the client, although the key benefit on the client is not in scaling, but in the ability to easily write responsive and fluid user interfaces. This is achieved by enabling the client application’s message pump to process messages while awaiting work. As a result, it is important for client apps to call WCF services asynchronously. WCF 4.5 enables that with the new Task-based async support. Let’s take a look at what the service consumption experience looks like in Visual Studio 2012, for the service above.

When you launch your WCF service, you’ll see the WCF Test Client. For the service above, a “wire shape” is exported to clients, as shown in the screenshot below. You’ll quickly notice that there is no mention of “Async” in the IService1 contract.

Let’s take a look at the client proxy that’s displayed after I add a reference to this service in a client app. Once again, you see that WCF delivers on the promise of enabling different implementation approaches within the client and service. In the screenshot below, you see that WCF generates synchronous and asynchronous methods for each of the methods in IService1, as shown in the WCF Test Client. As a client developer, I get to choose whether to call GetData or GetDataAsync, GetSomeData or GetSomeDataAsync. That approach enables great flexibility for client developers.

If you select F12 to go to references on IService1 or any of its members, you’ll see the client proxy code that was generated by WCF and will notice that the code exactly matches the shape that you see in the object browser.

Last, I want to write some code that calls into my new WCF service. You can see a simple button click event handler within a WPF app below. The code creates an instance of my service, and then proceeds to call the two methods exposed on the service. To make the point on flexibility, I opted to call one method synchronously and the other method asynchronously. If this were a real app, I’d likely make both client calls asynchronous; however, you get to make that choice yourself.

async void Button_Click_1(object sender, RoutedEventArgs e)
{
    Service1Client client = new Service1Client();

    // synchronous call on Service1
    String result = await client.GetDataAsync(1);
    // asynchronous call on Service1
    String result2 = client.GetSomeData(42);
}

In closing

The .NET Framework 4.5 is a very big step forward for cloud workloads. .NET Framework async, in particular, is very well suited to the cloud, and provides you with a new programming model for scaling your sites and services. Async can scale your site by making more efficient use of machine resources, while still enabling you to parallelize network and I/O operations. The new model is also a win from a coding perspective, since you have the full support of Visual Studio 2012 and the managed language and class libraries within the .NET Framework 4.5. Async helps you make better use of the machines or virtual machines that you are already using, without much effort on your part

The big news is that the .NET Framework 4.5 is now supported in Windows Azure. I know that a lot of you have been waiting for that! Simply deploy your apps on the Windows Azure services that we covered above: Cloud Services, Windows Azure Web Sites, and Virtual Machines. After you’ve deployed your app, you should consider taking advantage of the new async APIs in the .NET Framework 4.5, and the new Task-based asynchronous programming model. Async has an addictive nature to it. Once you start using it and realizing the benefits, you’ll want to use it everywhere. Go right ahead!

Follow us or talk to us on Twitter — https://twitter.com/dotnet.

Brandon Bray

Follow