Azure Storage - Essential Facts

August 2014 - Definitely subject to change

The platform is evovling quickly. This is a snapshot as of Aug 2014.

Utilities

The following section provides some guidance about using external utilities with Azure Storage.

Client Utilities

These are tools used by client applications that access or help with Azure Storage.

Copy Utilities - AzCopy

The fastest way to move data from Azure Blobs to Azure File is to use AzCopy. You should run AzCopy from a VM in the same datacenter as the destination storage account.

AzCopy is now in release 2.5 can can be found here:

https://aka.ms/downloadazcopy

Figure 1: Azure Command prompt

Note the directory where it can b found. I will copy the binaries to

Here is the syntax:

Figure 2: AzCopy Syntax

Here are some things to remember:

1. You can copy files that are in file system directory, a blob container, a blob virtual directory, or a storage file share.

2. You can copy recursively as well.

3. You can copy a single blob or multiple with wild-cards.

4. You can copy across storage accounts.

5. With geo-redundancy you can copy blobs from secondary regions

6. You can also copy snapshots to another storage account

7. You can use response files to support automation

8. You can use Shared Access signatures

9. Log files can be generated

10. Works in the storage emulator

Copying across storage accounts asynchronously enables scenarios, like:

1. Backup up blobs

2. Migrate blobs to different account

3. The asynchronous copy blob runs in the background using spare bandwidth capacity, so there is no SLA in terms of how fast a blob will be copied

4. Cross account copies involve an egress fee.

5. You can copy the binary to where they are needed:

Figure 3: Location of AzCopy Binaries

1. https://blogs.msdn.com/b/windowsazurestorage/archive/2012/06/12/introducing-asynchronous-cross-account-copy-blob.aspx for more information

2. To protect from the source changing, you can use a lease, introducing the concept of lock (i.e. infinite lease) which makes it easy for a client to hold on to the lease.

3. During a pending copy, the blob service ensures that no client requests can write to the destination blob

How you get charged

There are 3 ways you get charged for Azure Storage.

Replicating Data / Redundancy

Azure Storage lets you specify how data gets replicated. There are 4 tiers. They differ on how they work, how much they cost, and how they perform. You should also understand the difference between a primary and secondary region.

Primary and Secondary Regions

Table of Features regarding Redundancy Strategy

How to the failover scenarios work?

When a primary region goes down, how does Azure recover your data

Regarding blobs, what are the capacities?

This table describes the capacity, throughput, and max size of a blob.

Egress (data leaving a data center) Costs

It gets cheaper with scale. But it varies by region.

Questions about data leaving region

Transaction Costs

Every time you have read or write operations, there is a cost.

Azure Tables - Support for JSON

It used to be AtomPub. But JSON makes more sense for a variety of reasons.

1. JSON provides minimal metadata.

2. Dramatic reduction payload size, saving CPU cycles, supporting higher scale, lower latency

Azure has an extensive set of SDKs

Of course, Azure Storage supports REST, meaning that you can talk to storage from any client that can use HTTP. But more specialized SDKs are also available.

Performance of Geo Redundant versus Local Redundant

Note that GeoRedundant slows down your uploading and download of content to MS data centers. Note that GRS does not impact latency of transactions made to the primary location.

Tooling to Manage Azure Storage

These are some free client tools that will allow you interact with data using Azure Storage Services

Understanding Cross Origin Resource Sharing (CORS)

Microsoft does support Cross Origin Resource Sharing (CORS) - Why this is so important

1. This support makes it possible for client-side web applications running from a specific domain to issue requests to another domain

2. This will allow JavaScript code loaded as part of https://www.contoso.com to issue requests at will to any other domain like https://www.northwindtraders.com

3. If CORS were not supported, you'd have to use a proxy for storage calls, limiting scale and adding an extra layer of work.

4. CORS makes it possible for web apps to directly place content to Azure Storage from your company web site.

5. More specifically, your end users could directly upload blobs using shared access signatures to a company storage account without the need of a proxy service.

6. You can therefore benefit from the massive scale of the Windows Azure Storage service without needing to scale out a service in order to deal with any increase in upload traffic to your website.

7. It is about granting a Web Browser write privilege to your company's storage account

8. Your web service does not need to be in the upload path of storage services

9. It works very simply - whenever a user is ready to upload, the JavaScript code would request a blob SAS URL to upload against from your service and then perform a PUT blob request against storage

10. As a precaution, it is recommended that you limit the access time of the SAS token to the needed duration time in order to limit any security risks and the specific container and or blob to be uploaded

11. Another scenario where this is usesful is allowing users to edit data in a browser and persisting the data to Windows Azure Tables, which is a dictionary-like persistent store.

    • How to save data to Azure Tables from a browser
    • Extended sample on leveraging CORS

12. Here is some guidance on how to enable CORS

    • Enabling CORS
    • Using Client Library to enable CORS - Windows Azure Storage Client .NET Library 3.0

Conclusion

I hope that I have surfaced some key facts that are buried in blogs and in on-line documentation.