Performing Asynchronous Work, or Tasks, in ASP.NET Applications

In this post I hope to clear up some misconceptions about the use of threads in ASP.NET applications so that you know the best way to perform asynchronous work in your ASP.NET applications. If you’re in a hurry and don’t want to read the rest of this, then I suggest that you use PageAsyncTask to execute work asynchronously, and enjoy the rest of your day.

 

I explained how ASP.NET and IIS use threads in an earlier post titled “ASP.NET Thread Usage on IIS 7.0 and 6.0”. If you haven't read that, please do, but just so you're not left wondering, I will summarize the "flow" of an ASP.NET request again here: New requests are received by HTTP.sys, a kernel driver. HTTP.sys posts the request to an I/O completion port on which IIS listens. IIS picks up the request on one of its thread pool threads and calls into ASP.NET where ASP.NET immediately posts the request to the CLR ThreadPool and returns a pending status to IIS. Next the request is typically executed, although it can be placed in a queue until there are enough resources to execute it. To execute it, we raise all of the pipeline events and the modules and handlers in the pipeline work on the request, typically while remaining on the same thread, but they can alternatively handle these events asynchronously. When the last pipeline event completes, the response bytes are sent to the client asynchronously during our final send, and then the request is done and all the context for that request is released.

 

In case you're curious, the IIS thread pool has a maximum thread count of 256. This thread pool is designed in such a way that it does not handle long running tasks well. The recommendation from the IIS team is to switch to another thread if you’re going to do substantial work, such as done by the ASP.NET ISAPI and/or ASP.NET when running in integrated mode on IIS 7. Otherwise you will tie up IIS threads and prevent IIS from picking up completions from HTTP.sys. So for this reason, ASP.NET always returns a pending status to IIS, and calls QueueUserWorkItem to post the request to the CLR ThreadPool. In v2.0, 3.5, and 4.0, ASP.NET initializes the CLR ThreadPool with 100 threads per processor (that’s the default, this is configurable). So on a dual-core server, there will be a maximum of 200 threads in the pool. When a CLR ThreadPool thread becomes available (typically happens immediately), the request is picked up by ASP.NET. Normally the request is executed at this point, but it is possible for it to be inserted into one of the ASP.NET queues (described in earlier post). If all the modules and handlers in the pipeline are synchronous, the request will execute on a single CLR ThreadPool thread. If one or more modules or the handler are asynchronous, the request will not execute on a single thread. This is where your code comes into play, but first I have a little more information for you.

 

If you read the paragraphs above, you probably noticed that a lot of thread switches take place in order to execute an ASP.NET request. There are always at least 3 thread switches. The first is primarily a transition from kernel mode (HTTP.sys) to user mode (IIS). It also frees up HTTP.sys to pick up more incoming requests and hand them off to their respective listeners—not always IIS. If we tried to execute the entire request on that thread, HTTP.sys wouldn’t be resilient to poorly performing listeners—in fact, a single process could shutdown HTTP on the entire server if that process were to deadlock and hold the existing request threads as well as any new incoming request threads. At the same time, there is a penalty paid for any thread switch. It’s called a context switch, and they’re expensive. However, in this case, the benefit from performing the thread switch (reliability of a kernel mode driver) out weights the cost of the context switch. The second thread switch is the one from IIS to ASP.NET, where ASP.NET calls QueueUserWorkItem for the CLR ThreadPool. This one is not as critical. As mentioned in an earlier blog post, this is done primarily as a performance improvement for large corporate workloads that have a mixture of static and dynamic requests. The thread switch for dynamic requests helps improve CPU cache locality for static requests (served by the IIS static file handler) which don't peform a thread switch and instead just execute until completion on the IIS thread. Your specific workload may be quite different—the context switch may be more expensive than any performance benefit from performing the thread switch. I think the performance benefit is very subtle regardless, and a more important reason to perform the thread switch is to free IIS threads so IIS can pick up more completions and hand requests off to other applications—not always ASP.NET. This is similar to what we described in the Http.sys case. We decided it was better to perform the thread switch than not, and so that's what we do. The third thread switch is the one that occurs when we perform the final send, when we switch from a CLR ThreadPool thread to the thread that send the bytes to the client. This switch is definitely worth the cost of the context switch, because clients with low bandwidth are slow and we don't want our CLR ThreadPool thread to be blocked while the bytes are sent, we want it to return to the pool and execute incoming work items. Ok, back to your code..., lets discuss your code now.

 

So, in your ASP.NET application, when should you perform work asynchronously instead of synchronously? Well, only 1 thread per CPU can execute at a time. Did you catch that? A lot of people seem to miss this point...only one thread executes at a time on a CPU. When you have more than this, you pay an expensive penalty--a context switch. However, if a thread is blocked waiting on work...then it makes sense to switch to another thread, one that can execute now. It also makes sense to switch threads if you want work to be done in parallel as opposed to in series, but up until a certain point it actually makes much more sense to execute work in series, again, because of the expensive context switch. Pop quiz: If you have a thread that is doing a lot of computational work and using the CPU heavily, and this takes a while, should you switch to another thread? No! The current thread is efficiently using the CPU, so switching will only incur the cost of a context switch. Ok, well, what if you have a thread that makes an HTTP or SOAP request to another server and takes a long time, should you switch threads? Yes! You can perform the HTTP or SOAP request asynchronously, so that once the "send" has occurred, you can unwind the current thread and not use any threads until there is an I/O completion for the "receive". Between the "send" and the "receive", the remote server is busy, so locally you don't need to be blocking on a thread, but instead make use of the asynchronous APIs provided in .NET Framework so that you can unwind and be notified upon completion. Again, it only makes sense to switch threads if the benefit from doing so out weights the cost of the switch.

 

Ok, so you have a good reason to perform some work asynchronously, how should you do it? First of all, all of the code that you are able to run during the execution of a request must run within a module or handler. There is no other option. If you want work to be performed asynchronously—truly asynchronously, as in the current thread unwinds and execution of the request resumes only if and when your work completes—then you must run inside a module or handler that is asynchronous. If you don’t want to implement your own asynchronous module or handler, you’re in luck, because ASP.NET 2.0 introduced async pages , a feature which builds upon IHttpAsyncHandler and makes it easy to run asynchronous tasks known as PageAsyncTasks. There is a nice introduction to asynchronous pages in Dmitry’s TechEd presentation. I would think most ASP.NET developers would prefer to use async pages with one or more PageAsyncTasks in order to perform work asynchronously. So I encourage you to read Dmitry’s TechEd presentation, and focus on slides 31 and 32.

 

If instead you would prefer to write your own asynchronous module or handler, I’ve included sample code for an asynchronous IHttpModule below along with the same code for a synchronous version of that IHttpModule. Implementing an IHttpAsyncHandler is very similar, so I didn't bother to provide an example.

 

"AsyncModule.cs":

 using System;
using System.Web;
using System.Threading;

public class AsyncModule: IHttpModule {

    // IHttpModule implementation
    void IHttpModule.Dispose() {}

    void IHttpModule.Init( HttpApplication app ) {
        app.AddOnPreRequestHandlerExecuteAsync(new BeginEventHandler(OnBegin),
                                               new EndEventHandler(OnEnd)); 
    }

    // Post a work item to the CLR ThreadPool and return an async result 
    // with IAsyncResult.CompletedSynchronously set to false so that the 
    // calling thread thread can unwind.
    private IAsyncResult OnBegin(Object sender, 
                                 EventArgs e, 
                                 AsyncCallback cb,
                                 Object state) {
        IAsyncResult ar = new MyAsyncResult(cb, 
                                            ((HttpApplication)sender).Context);
        ThreadPool.QueueUserWorkItem(DoAsyncWork, ar);
        return ar;
    }

    // Invoked after completion of the event.  This is frequently a no-op, but
    // if you need to perform any cleanup you could do it here.
    private void OnEnd(IAsyncResult asyncResult) {
    }

    // Called by CLR ThreadPool when a thread becomes available.  We must call
    // Complete on the async result when we are done.  When this happens, the
    //  callback passed to OnEnter is invoked to notify ASP.NET that we are done 
    // handling the event. ASP.NET will then resume execution of the request.
    private void DoAsyncWork(Object state) {
        MyAsyncResult ar = state as MyAsyncResult;
        try {
            throw new NotImplementedException(
                "REMOVE THIS AND ADD YOUR CODE HERE");
        }
        catch(Exception e) {
            ar.Context.AddError(e);
        }
        finally {
            // we must notify ASP.NET upon completion
            ar.Complete(false);
        }
    }

    public class MyAsyncResult: IAsyncResult {
        private AsyncCallback  _callback;
        private HttpContext    _context;
        private bool           _completed;
        private bool           _completedSynchronously;

        // IAsyncResult implementation
        bool IAsyncResult.IsCompleted { get { return _completed;} }
        bool IAsyncResult.CompletedSynchronously { 
            get { return _completedSynchronously; } 
        }
        Object IAsyncResult.AsyncState { get { return null;} }
        //wait not supported
        WaitHandle IAsyncResult.AsyncWaitHandle { get { return null;} }

        // store HttpContext so we can access ASP.NET intrinsics 
        // from our DoAsyncWork thread
        public HttpContext  Context { 
            get { 
                if (_completed || _context == null) {
                    // HttpContext must not be accessed after we 
                    // invoke the completion callback
                    throw new InvalidOperationException();
                }
                return _context; 
            }
        }

        public MyAsyncResult(AsyncCallback cb, HttpContext context) {
            _callback = cb;
            _context = context;
        }

        public void Complete(bool synchronous) {
            _completed = true;
            _completedSynchronously = synchronous;
            // HttpContext must not be accessed after we invoke the 
            // completion callback
            _context = null;
            // let ASP.NET know we've completed the event
            if (_callback != null) {
                _callback(this);
            }
        }
    }
}

 

"SyncModule.cs":

 using System;
using System.Web;

public class SyncModule: IHttpModule {

    // IHttpModule implementation
    void IHttpModule.Dispose() {}

    void IHttpModule.Init( HttpApplication app ) {
        // There are many events, but here we're using 
        // PreRequestHandlerExecute
        app.PreRequestHandlerExecute += new EventHandler( 
                                            OnPreRequestHandlerExecute );
    }

    private void OnPreRequestHandlerExecute(object sender, EventArgs e) {
        HttpApplication app = sender as HttpApplication;
        throw new NotImplementedException(
                     "REMOVE THIS AND ADD YOUR CODE HERE");
    }
}

 

FAQ

 

Q1) How many thread pools are there?

 

A1) There is only one managed thread pool: the CLR ThreadPool. This is the thread pool that is used by the .NET Framework. However, using Win32 APIs you are free to implement as many thread pools as you like. Almost nobody does this because it is very difficult to get it right.

 

Q2) When should I perform work asynchronously?

 

A2) When the benefit of switching threads out weights the cost of the context switch. In an attempt to simplify this decision, you should only switch if you would otherwise block the ASP.NET request thread while you do nothing. This is an oversimplification, but I'm trying to make the decision simple. For example, if you make an asynchronous web service request to a remote server then it would be better to let the ASP.NET request thread unwind, and when the "receive" for the web service request fires your callback, you would call into ASP.NET allowing it to resume exection of the pipeline. That way the ASP.NET request thread is not blocked doing nothing while you wait for the web service request to complete.

 

Q3) If my ASP.NET Application uses CLR ThreadPool threads, won’t I starve ASP.NET, which also uses the CLR ThreadPool to execute requests?

 

A3) Think about it this way: if you stay on the same thread, you’re using one of ASP.NET’s threads. If you switch to another ThreadPool thread and let ASP.NET’s thread unwind, you’re still only using one thread. While it is true that ASP.NET in classic mode will queue requests if there are more than 12 threads per CPU currently in-use (by default, see KB 821268), for most scenarios you don’t want more than 12 threads to be executing at any given time. When you’re doing asynchronous work, you’re not always using a thread. For example, if you made an async web service request, your client will not be using any threads between the “send” and “receive”. You unwind after the “send”, and the “receive” occurs on an I/O completion port, at which time your callback is invoked and you will then be using a thread again. (Note that for this scenario your callback is executed on an i/o thread—ASP.NET only uses worker threads, and only includes worker threads when it counts the threads in-use.) This is a juggling game, and you will see that when the requests are asynchronous, there can be more requests "executing" at any given time then the number of threads currently in-use. IIS 7 integrated mode is different, in that ASP.NET does not limit the number of threads in-use by default—but you do need to set MaxConcurrentRequestsPerCPU = 5000 in v2.0/3.5, as I mentioned in another post. So to summarize, don’t worry about starving ASP.NET of threads, and if you think there’s a problem here let me know and we’ll take care of it.

 

Q4) Should I create my own threads (new Thread)? Won’t this be better for ASP.NET, since it uses the CLR ThreadPool.

 

A4) Please don’t. Or to put it a different way, no!!! If you’re really smart—much smarter than me—then you can create your own threads; otherwise, don’t even think about it. Here are some reasons why you should not frequently create new threads:

 

1) It is very expensive, compared to QueueUserWorkItem.

 

2) Before you allow your thread to exit, you must check for pending I/O because your new thread might call into a .NET Framework API that initiates an asynchronous operation, and there’s really no way for you to know whether or not this happened unless you don’t call into framework APIs.

 

3) The performance of the system hinges on the fact that only a reasonable number of threads are executing at any given time, and if you start creating your own threads it will then become your responsibility to maintain performance of the system. By the way, if you can write a better ThreadPool than the CLR’s, I encourage you to apply for a job at Microsoft, because we’re definitely looking for people like you!

 

Q5) What if I want to perform work in the background, independent of any requests?

 

A5) Do you really need it to be independent of a request? It really doesn’t matter how long it runs, it can still be run in the context of a request. The problem with performing work independent of a request is that the AppDomain can be unloaded, and any threads executing at that time will be rudely aborted, which could cause you quite a bit of grief depending on what you’re doing. In v4.0 ASP.NET, we won’t unload the AppDomain until all requests have completed. In v2.0/3.5 ASP.NET in IIS 7 integrated mode, we also won’t unload the AppDomain until all the requests have completed. In v2.0/3.5 classic mode, we will only wait until httpRuntime/shutdownTimeout has been exceeded before unloading the AppDomain, so long running async requests are vulnerable to being rudely aborted. To work around this, you’d want the shutdownTimeout to be sufficiently long.

 

If you really want this asynchronous work to be independent of any requests, then I recommend that you register for a notification that will tell you when the domain is about to shutdown so that you can safely complete/timeout the work. There are three ways to do this:

1) Use the ApplicationEnd event.

2) There is an AppDomain.DomainUnload event which is fired immediately before the AppDomain is unloaded.

3) You can create an object that implements the IRegisteredObject interface and call HostingEnvironment.RegisterObject to “register” it with ASP.NET. When the AppDomain is about to be unloaded, we will call your implementation of IRegisteredObject.Stop(bool immediate). The Stop method is called twice by ASP.NET, once with the immediate parameter set to false and once again with that argument set to true. You are supposed to call HostingEnvironment.UnregisterObject as soon as your registered object has stopped, so ASP.NET knows it doesn’t need to call your Stop method. You can call it at anytime, but you definitely should call it before returning from Stop when it is called with immediate set to true, because that’s your final chance and if you’re still running after that you will be rudely aborted. This functionality was not implemented specifically for the ability to execute long running tasks independent of any requests, but it will enable you to do this and safely shutdown/complete your work. Note that the point of calling Stop twice is to allow you to asynchronously kick off a job to stop your long running task when Stop is called the first time, giving you a bit more time before it is called the second time and you have to shutdown. If you need to, you can hold up the unload as long as you like, because we won’t unload until your Stop method returns the second time. However, you should be well behaved, and return in a reasonable amount of time, and preferably immediately the first time Stop is called.

 

Q6) You didn't answer my question.

 

A6) To be fair, I don't know what your question is. I don't have anonymous comments enabled (too much spam), but if you take a few seconds to register you can leave comments below and I will answer them.

 

Thank you,

Thomas