Collecting Windows Azure Storage REST API level metrics data without a single line of programming, just by using tools

  • Article

Time to time you might want to know what the total transaction counts are when you perform any storage activity which included Windows Azure storage. 

For example when you mount a VHD from Windows Azure Storage to you Azure VM or you write total X number of file included Y MB/GB of data. I decided to write this article to answer all of these questions.

First you really need to understand how Windows Azure Billing works and there is no better place then the link below:

https://blogs.msdn.com/b/windowsazurestorage/archive/2010/07/09/understanding-windows-azure-storage-billing-bandwidth-transactions-and-capacity.aspx

The above link explains how Windows Azure Billings works along with few examples as below:

  • A single GetBlob request to the blob service = 1 transaction
  • PutBlob with 1 request to the blob service = 1 transaction
  • Large blob upload that results in 100 requests via PutBlock, and then 1 PutBlockList for commit = 101 transactions
  • Listing through a lot of blobs using 5 requests total (due to 4 continuation markers) = 5 transactions
  • Table single entity AddObject request = 1 transaction
  • Table Save Changes (without SaveChangesOptions.Batch) with 100 entities = 100 transactions
  • Table Save Changes (with SaveChangesOptions.Batch) with 100 entities = 1 transaction
  • Table Query specifying an exact PartitionKey and RowKey match (getting a single entity) = 1 transaction
  • Table query doing a single storage request to return 500 entities (with no continuation tokens encountered) = 1 transaction
  • Table query resulting in 5 requests to table storage (due to 4 continuation tokens) = 5 transactions
  • Queue put message = 1 transaction
  • Queue get single message = 1 transaction
  • Queue get message on empty queue = 1 transaction
  • Queue batch get of 32 messages = 1 transaction
  • Queue delete message = 1 transaction

Now please follow given steps to collect Windows Azure Storage REST API level these details:

Step 1: Download CloudBerry Explorer for Windows Azure Version 1.4.1 +
https://www.cloudberrylab.com/microsoft-azure-explorer-pro.aspx

Step 2: Now create a Windows Azure Storage Service and configured in CloudBerry Azure Blob Storage Explorer. After configuration please enabled Storage Metrics as below:

2.1 Open menu Analytics > Settings:

2.2 Select your Azure Storage Name:

2.3 Now enabled logging as below: (You can also select “Delete data older then X days” as well if want)

2.4 Now Metrics data as below: (You can also select “Delete data older then X days” as well if want)
 

2.5 Select Apply to confirm all of above settings.

Step 3: Now for testing please copy a file in your above configured Windows Azure Storage. As you can see below I have copied 64MB VHD to my Azure Storage.

Step 4: Now let’s check the Statistics for just copy operation.

4.1 Go to Analysis > View Log as below:
 

4.2 You will a long list of table for each Storage API used for the transaction. If log is not visible please try to set the Date Interval correctly.

Log Analysis:

You can also copy/save the log and Open in TextAnalysisTool.NET (Download from this link) to parse as below:

Or you can use Microsoft Excel 2010 to filter the log file as below to get API specific details:

This way you can get API Specific metric data for your Azure Storage.