Azure Data Services: Designing a communication strategy using Storage Queues (Part 5)

Hello and welcome to this tutorial series of Azure Data Services. Through the course of this tutorial, I will take you through what Azure has to offer in Data Services. I will start with the basics of relational databases, dig into the goodness of non-relational offerings and end with what is new and what is exciting in Azure Data Services

Part 1: An Introduction to Azure Data Services

Azure Data Services: SQL in the Cloud (Part 2)

Azure Data Services: Blobbing did you say.. What is that? (Part 3)

Azure Data Services: NoSQL Table Storage (Part 4)

Communication between applications or between the different modules in an application is often key to decoupled (or loosely coupled) programming practices. Azure Queues are a means of storing messages for later consumption. This is accomplished by either using Azure Storage Queues or Azure Service Bus. Both are similar, as in both are queuing mechanisms but differ in the way they are implemented which also implies the use cases are different. The table below lists out the differentiation between the two queuing mechanisms:

In this article, I will focus on Azure Storage Queues and discuss use cases and how we use the queues. Essentially Azure Queue Storage Service is a persistent message storing and retrieving service. It is also built for massive scale and resilience. Each message can be upto 64KB in size and can store millions of message till the maximum capacity of the storage account of 200 TB (A Storage Account can be a combination of Tables, Blobs and Queues).

A storage account may contain one of many queues and a queue can contain one or many messages. The most popular use cases for Azure Queue is to communicate between Azure Web and Worker roles. It is also used to process asynchronous workloads.

Before we get started with some demo, let us have a look at some interesting characteristics and operations that pertain to our queue:

• Add/Delete Messages: We can add and delete messages from our queue which corresponds to insert and de-queue(make invisible for a time period) operations
• Get Message from the queue implies that you have retrieved the message from the queue and making it “invisible” to others for a timeperiod
• Peek at the next message which is on the queue implies you have retrieved it but is still “visible” to others
• Update Messages can allow for you to change the timeout period on a message on a queue
• Since Storage Queues does not ensure a maximum retrieval of once, every operation on the message should be idempotent, which implies that every operation should ensure the same result

Lets get started with creating and playing with queues:

Step 1: Create Storage Account

1. Login to Azure Preview Portal: portal.azure.com

2. Create Storage Account: New-> Everything –> Storage, Cache –>Storage-> Create

Step 2: Access your storage account, play with queues using Visual Studio:

1. Let us open a blank console in Visual Studio. Add the Azure Storage Nuget Package and the following ‘using’ statements:

    1: using Microsoft.WindowsAzure.Storage;
    2: using Microsoft.WindowsAzure.Storage.Auth;
    3: using Microsoft.WindowsAzure.Storage.Queue;

2. Now let us create an object to access the storage account created in Step 1

    1: CloudStorageAccount account = new CloudStorageAccount(new StorageCredentials("<storage account name", 
    2:                 "<Storage Account Primary Access Key>"), 
    3:                 true);

3. Let us create the queue client through whom we will be accessing the storage account:

    1: var queueclient = account.CreateCloudQueueClient();
    2:             var queue = queueclient.GetQueueReference("myqueue");
    3:             queue.CreateIfNotExists();

4. Now is the fun time, let us add some messages in our queue:

    1: queue.AddMessage(
    2:                 new CloudQueueMessage( "Message 1")
    3:                 );
    4:  
    5:             queue.AddMessage(
    6:                 new CloudQueueMessage( "Message 2")
    7:                 );

5. After executing this simple program, if we look through the server explorer, we will notice two messages in our queue which was just created:

6. Now let us read these messages:

    1: var msg = queue.GetMessage();
    2:             Console.WriteLine(msg.AsString);
    3:             Console.ReadLine();

7. On executing this once, you will notice that one message remains, as one is consumed in the queue:

8. You can conduct additional operations such as peek too:

    1: var msg = queue.PeekMessage();
    2:             Console.WriteLine(msg.AsString);
    3:             Console.ReadLine();

9., Again, you can go to the queue and notice that the message still remains in the queue as we just “peeked” into the queue.

10. If you wait for sometime, and then refresh the queue you will notice that both the messages will be back and this is because we did not delete the messages from the queue and it was back as the “invisible” time on message one expired and so it reappeared. Hence a good practice is when you delete the message when you read it:

    1: var msg = queue.GetMessage();
    2:             queue.DeleteMessage(msg);
    3:             Console.WriteLine(msg.AsString);

That's it, those are some of the operations that you can do on a Azure Storage Queue. It is quite a simple construct and is very handy to develop loosely coupled applications which involves some form of communication as in the case between Azure WebRole and WorkerRole.

Summary

We started by describing how communication can be accomplished in Azure, namely using Azure Storage Queues and Azure Service Bus. We differentiated each of them briefly and then focused on Storage Queues. We covered the basic operations, functionality and programming constructs that are involved with the queue. If you need any more information, you can look into How to Use Queues. To get started with Azure Service you can look into How to Use Service Bus Queues.

After having covered queues, I have covered the three kind of Data Storage Services in Azure through a Storage Account. I will be covering a few more interesting Data Services in Azure in the coming weeks, so stay tuned. Would love to get feedback and listen to your experiences with Azure. Tweet me @AdarshaDatta

Technorati Tags: Cloud,Data Storage,Queue,CloudDev,Data Queue,FIFO,PaaS