Correctly handling InvalidOperationException thrown by CommitAsync


The CommitAsync method in the Reliable Collections programming model can throw InvalidOperationException in a couple of cases:

  • When the transaction has already been committed
  • When the transaction has been aborted either by the system or by the user

In the v5.2 runtime (to be released soon), a series of stability improvements have increased the number of system-aborted transactions, thus increasing the likelihood of CommitAsync throwing an exception. To correctly handle these exceptions, make sure you're doing the following in your code:

  1. Dispose the transaction
  2. If you are using RunAsync, check if the cancellation token is signaled. If it is, exit by calling ThrowIfCancellationRequested on the token. Otherwise, back off and retry the transaction.


The following code snippet shows an example of how to handle InvalidOperationException in a background work function:

async Task BackgroundWorkAsync(CancellationToken cancellationToken)
{
  int retryCount = 0;

  while (true)
  {
    using (ITransaction tx = this.StateManager.CreateTransaction())
    {
      try
      {
        await reliableDictionary.SetAsync(tx, key, value).ConfigureAwait(false);
        await tx.CommitAsync().ConfigureAwait(false);
        return;
      }
      catch(TimeoutException)
      {
        // allow transaction to be disposed and retry
      }
      catch(InvalidOperationException)
      {
        // allow transaction to be disposed and retry
      }
      catch(FabricNotPrimaryException)
      { 
        // Exit the loop and proceed with failover
        return; 
      }
    }

    cancellationToken.ThrowIfCancellationRequested();

    // BackOffAsync method not shown
    await this.BackOffAsync(retryCount++).ConfigureAwait(false);
  }
}


If you have questions, please post them as comments below.

Cheers,

The Service Fabric Team


Comments (4)

  1. Artem says:

    Do we really can just throw an exception by calling cancellationToken.ThrowIfCancellationRequested() and do not handle it? Or we should handle it somehow in the calling code? If we should handle it, is there any special things that we need to take into account?

    1. Vaclav Turecek says:

      Artem, yes you can do cancellationToken.ThrowIfCancellationRequested(). The runtime will catch this and compare the token to the token provided to you in RunAsync. If they match, all is well and the runtime accepts it as a graceful cancellation.

  2. Rasmus says:

    What scale of time is we talking about in the BackOffAsync method - is it a few ms or a few sec ?

    What its good practices to do about the FabricNotPrimaryException when your code is trigger from a remote call, so your transactions is not in the "override Task RunAsync" ?

Skip to main content