Using Azure Queues to schedule work items


Attached sample shows how to use Azure Storage Queues to schedule processing of an item for a later point in time. It solves a common problem of scaling out a process based on iterating over a collection and processing some items based on time specific properties. Such an iteration, while easy enough to implement in a single process becomes difficult when implemented in multiple parallel processes. Using Azure Storage Queues (Azure Service Bus Queues have similar facilities) one can create messages which can only be accessed at a specific future point in time (within 7 days for Azure Queues). This way multiple processes reading the queue can execute in parallel in a time-based sequence.

The sample consists of two console projects: a sender and a receiver. The sender creates 10 messages scheduled for processing with up to 10 minutes delay, randomly distributed among the messages. After the messages are sent, they are listed in the sequence they should be received. The second projects runs an endless loop waiting for messages and displays them in the sequence they arrive. The two displays should match (mostly: Azure Storage Queue, unlike ServiceBus queues do not guarantee sequence of delivery).

QueueMsgScheduling.zip

Comments (2)

  1. GabiKliot says:

    Sorry, but this is a really low quality code sample:

    1) Does not compile. Should have included dependent assemblies (Azure storage client library assemblies) in the sample.

    2) No error handling code.

    3) Bug in usage of Task.Delay. The returned task is not awaited or waited upon, as a result the call to Task.Delay is doing nothing, not generating the actual delay in execution.

    4) Usage of synchronous APIs, while a good and efficient code should use asynchronous APIs.

    A better tutorial: http://www.windowsazure.com/…/queue-service

  2. mrochon says:

    @GabiKliot: thanks for your comments. This was not intended as a general Azure queuing sample but rather to show the use of a feature of queues to scale a time-dependent workload. The sample does require Azure Storage SDK, which I should have mentioned. Yes, the use of Task.Delay is incorrect – it should have been Task.Delay(1000).Wait(). Thanks again.