Async CTP Refresh – design changes


Async CTP Refresh – Design Changes

The big news about the Async CTP Refresh is that it enabled development on SP1 and for Windows Phone 7, and came with a new EULA. But there were also a few design changes…

 

Async is like the zombie virus

I’ve come to believe that async will be like the “zombie virus” – once it bites one part of your program, you’ll be inclined make more of your program async just in case an operation takes time. But normally your async methods will complete immediately — and we’ve made some design changes to help the performance of this “fast path”.

 

Design change: new “await” pattern for greater efficiency

In the Async CTP Refresh, we changed the “await” pattern to make the fast path more efficient. This will affect anyone who made their own types awaitable. The following code is in VB, but it applies equally to C#.

‘ What pattern does the compiler use to implement an await expression?
Dim i = Await e

‘ Old pattern, in first Async CTP

Dim
$temp = e.GetAwaiter()
SAVE_STATE()
If $temp.BeginAwait(AddressOf cont) Then
    Return
End If
cont:
RESTORE_STATE()
Dim i = $temp.EndAwait()

‘ New pattern, in Async CTP Refresh
 
Dim $temp = e.GetAwaiter()
If Not $temp.IsCompleted Then
    SAVE_STATE()
    $temp.OnCompleted(AddressOf cont)
    Return
    cont:
    RESTORE_STATE()
End If
Dim i = $temp.GetResult()
i = Nothing

In the old pattern, it incurred the cost of SAVE_STATE() even if the task had already completed. We couldn’t put SAVE_STATE() inside the “Then” clause, because of the off chance that cont might be executed even before the “Then” clause had executed. In the new pattern we separated it out, so SAVE_STATE() and RESTORE_STATE() are never even executed in the “fast path”.

In the old pattern, it also incurred the cost of constructing a delegate (which I’ve written “Addressof <label>” in the pseudo-code above), even if it wasn’t needed. The new pattern will allow us to allocate that delegate lazily, potentially saving a heap allocation in the fast path. However, this is just potential for the future: we don’t actually take advantage of it.

In the old pattern, it also left the “awaiter” field hanging around. In the new pattern, we null it out — the C# equivalent is “default(T)”. This is so that the awaiter does not hold onto references for longer than necessary, which could harm garbage collection.

GUIDANCE

 

If you are concerned about garbage collection in a long-running async method, you can “null out” your local variables. This is the same as what you can currently do in long-running non-async methods.

 

Taking advantage of async “fast path”

How might you take advantage of the async fast path? Here’s some sample code, this time in C#. It uses the new method TaskEx.FromResult<T>() which was added in the CTP Refresh.

// This will use the “fast path” in the common case where the

// data is already available locally. It will avoid heap allocations

// in that case.

Database db;

while (await db.MoveNextAsync())

{

    Console.WriteLine(db.Current);

}

Thanks to the change, this “while” loop incurs only two small method calls (and no heap allocations) beyond what would be needed in a non-async alternative. The performance boost comes down to this implementation:

 

class Database

{

    private static Task<bool> trueTask = TaskEx.FromResult(true);

    private string[] currentBuffer = null;

    private int currentIndex = -1;

 

    public string Current {get {return currentBuffer[currentIndex];}}

 

    public Task<bool> MoveNextAsync()

    {

        if (currentBuffer != null && currentIndex < currentBuffer.Length-1)

        {

            currentIndex++;

            return trueTask; // avoids the cost of allocating a task

        }

        return MoveNextAsyncInternal();

    }

 

    public async Task<bool> MoveNextAsyncInternal()

    {

        currentBuffer = DownloadNextChunkFromDatabase();

        if (currentBuffer == null) return false;

        currentIndex = 0;

        return true;

    }

}

 

 

 

Costs of async… where even the “fast path” doesn’t help

Even if the fast paths are taken everywhere, there are still some inherent costs in async:

‘ Compare the cost of these two statements.

‘ The async one has some overhead…

Dim i1 = f1()

Dim i2 = Await f2()

 

Function f1() As Integer

    Return 1

End Function

 

Async Function f2() As Task(Of Integer)

    Return 1

End Function

  • The async call costs THREE extra heap allocations – one for the async state machine, one for its continuation delegate, and one for the resultant Task object.
  • The async call costs several extra method calls to set up those objects and those fields.
  • The async call costs additional IL instructions and an additional try/catch block within the body of the method.
  • The return statement in the async case costs an additional method call.
  • To await f2() costs two extra method calls.

 

So how is it that the previous code managed to avoid most of those costs?

while (await db.MoveNextAsync()) {

    Console.WriteLine(db.Current);

}

Here “MoveNextAsync()” was not actually an async method, i.e. it didn’t use the async modifier. And it returned a pre-allocated Task object instead of allocating a new one each time. In this way it bypasses most of the extra async costs. The only extra cost it incurs over and above the synchronous case is two extra method calls.

GUIDANCE

 

Create “chunky async” APIs rather than “fine-grained async” APIs. For instance, create APIs which asynchronously retrieve a batch of rows from the database in one call, rather than just one row at a time.

 

The extra cost of async is negligible compared to the latency of network operations or UI operations. It is only ever worth thinking about in inner-loops or in server code where you’re optimizing for scalability. As always, measure performance before optimizing.

 

Don’t make your code async just for the sake of it. Only do so for a reason, e.g. to avoid blocking the UI, or because the API calls you’re making are async, or to avoid consuming too many threads.

 

As in the above case, the extra costs of consuming a “fast-path” async can be minimized in some situations. The extra costs of producing an async method cannot.

 

In case of the zombie virus, it’s best to prepare an emergency kit beforehand. Include a shovel.

 

 

 

Design change: new exception behavior for “Async Subs”

We made another change because we wanted uniformity in the behavior of exceptions in Async Subs (“void-returning asyncs” in C#).

This hopefully won’t affect anyone!

GUIDANCE

 

It’s fine to use async methods that are Subs (void-returning asyncs) for top-level event handlers and the like. It’s okay for these to throw exceptions

 

It’s fine to use async methods that are Task-returning or Task<T>-returning, and have the suffix “Async”, for your normal async methods.

 

As for other acceptable uses of async methods, there are a few niche cases where it makes sense to write “fire-and-forget” Async Subs which the caller is never able to await, but these should not throw exceptions that your program is intended to handle.

For everyone who followed that guidance, the change in behavior of exception-throwing Async Subs won’t have any effect. That’s because the only exception-throwing Async Subs were the top-level event handlers.

 ‘ First Async CTP: idiosyncratic exceptions

Async Sub f()
    Throw New Exception(“A”)
    Await t
    Throw New Exception(“B”)
    Await TaskEx.Yield()
    Throw New Exception(“C”)
End Sub

‘ [A] Always thrown to the caller of f()
‘ [B] Might be thrown to caller of f(),
‘ or maybe to the caller of the
‘ continuation-after-t (usually the UI
‘ message pump), depending on whether t
‘ took the fast path or not
‘ [C] Always thrown by whoever called the
‘ continuation-after-Yield

 ‘ Async CTP Refresh: uniform exceptions

Async Sub g()
    Throw New Exception(“A”)
    Await t
    Throw New Exception(“B”)
    Await TaskEx.Yield()
    Throw New Exception(“C”)
End Sub

‘ [A] Thrown to the caller’s Sync.Context
‘ [B] Thrown to the caller’s Sync.Context
‘ [C] Thrown to the caller’s Sync.Context

Implementation in first Async CTP:

  1. At the start of the async method, the compiler implicitly generates a call to
    System.Runtime.CompilerServices.
    VoidAsyncMethodBuilder.Create()
  2. This saves the current SynchronizationContext.Current
  3. It then calls sc.OperationStarted()

  4. When the method completes normally, the compiler implicitly generates a call to
    builder.SetCompleted().
    This calls sc.OperationCompleted()
  5. If the method completed due to an exception, the compiler implicitly generates a call to
    builder.SetCompleted().
    This calls sc.OperationCompleted().
    Next, the compiler lets the exception propagate up the callstack.

Implementation in CTP Refresh:

  1. At the start of the async method, the compiler implicitly generates a call to
    System.Runtime.CompilerServices.
    AsyncVoidMethodBuilder.Create()
  2. This saves the current
    SynchronizationContext.Current
  3. It then calls sc.OperationStarted()

  4. When the method completes normally, the compiler implicitly generates a call to
    builder.SetCompleted().
    This calls sc.OperationCompleted()
  5. If the method completed due to an exception, the compiler implicitly generates a call to
    builder.SetException(ex).
    This calls sc.OperationCompleted().
    It then does sc.Post( () => throw ex ).

 

Comments (7)

  1. tobi says:

    What does the change mean to argument validation? I think it means that those exceptions will not appear directly at the call site but at some other random place, like on the message loop in WinForms. Isn't that hard to diagnose?

  2. Tobi – for "Argument Validation", let's look at how a regular event-handler vs an async event handler will work…

    Sub Button1_Click() Handles Button1.Click

      If String.IsNullOrEmpty(TextBox1.Text) Then Throw New Exception("Please supply TextBox1")

      …

    End Sub

    vs

    Async Sub Button1_Click() Handles Button1.Click

      Await TaskEx.Yield()

      If String.IsNullOrEmpty(TextBox1.Text) Then Throw New Exception("Please supply TextBox1")

    End Sub

    In both cases, it's the winforms message-loop that dispatched the Button1Click.

    In the first case, the exception gets sent to the caller, which in this case is the winforms message-loop.

    In the second case, the exception gets posted to the current synchronization context, which in this case is the winforms message loop.

    So in both cases they end up on the winforms message-loop, and the UI experience is the same.

    As for the debugging experience? We're still fine-tuning this. But an exception thrown from an async sub will almost certainly count as "unhandled by user code", giving VS the opportunity to stop right there at the moment it's thrown.

  3. tobi says:

    You are right with handlers but what if you wanted to kick of an async "workflow" by starting a void-returning method. An example would be AnimateElement(UIElement e). If the element e was null, I would prefer the exception being thrown right after the call happened so I could track down where the null ref came from. Maybe exceptions could be thrown directly up to the first async-point (because that code will execute sequentially anyway).

  4. Alex Corrado says:

    Lucian, thank you for sharing so many details about the development of the Async CTP. I started my own async method framework before the CTP was first announced, and it has been fascinating to compare your implementation choices with my own. In the example you gave above about the "Costs of async," you had this method:

    Async Function f2() As Task(Of Integer)

       Return 1

    End Function

    You seem to indicate that the above would still be transformed into a state machine. Why not instead add an implicit conversion between T and a completed Task<T>? Then, only perform the full transformation when the async method actually awaits something. You could save 2 of the extra heap allocs you mentioned.

  5. Alex, that's a nice idea. But note that "f2" actually will emit a warning ("Warning: this async method lacks awaits. Consider running it on a background thread using await Task.Run, or awaiting the async version of some APIs")… it didn't really seem worth optimizing code that generated a warning!

  6. Richard says:

    I think there's a small typo in the first example:

    "Dim i = $temp.GetResult()

    i = Nothing"

    Surely the second line should be "$temp = Nothing"?

  7. Glenn says:

    > Create “chunky async” APIs rather than “fine-grained async” APIs. For instance, create APIs which asynchronously retrieve a batch of rows from the database in one call, rather than just one row at a time.

    > The extra cost of async is negligible compared to the latency of network operations or UI operations.

    This says to create coarse APIs that cause fewer total async operations to occur than network operations, then goes on to say that the cost of the async option is negligible compare to network operations.  These are contradictory–which is it?