Supporting Billions of entities/rows for Mobile – Android Series - Part 3–Architecture and Data Options

Article
09/28/2011

Previous Posts to this post

Previous Blog Posts	Link
Supporting Billions of entities/rows for Mobile – Android Series - Part 1 - Why Scale Matters	https://blogs.msdn.com/b/brunoterkaly/archive/2011/09/27/supporting-billions-of-entities-rows-for-mobile-android-series-part-1-why-scale-matters.aspx
Supporting Billions of entities/rows for Mobile – Android Series - Part 2 - What are some high level cloud offerings?	https://blogs.msdn.com/b/brunoterkaly/archive/2011/09/27/supporting-billions-of-entities-rows-for-mobile-android-series-part-2-what-are-some-high-level-cloud-offerings.aspx
Free download link for Azure

Let’s talk architecture

At this point we are ready to write some code. Let’s just show a couple of diagrams to kind of give you the picture about what we are trying to do. What we are talking about is the “Service Provider” section. We are about to create a web service in the “Windows Azure” section of the cloud diagram below. This diagram depicts an easy way to think about the style of architecture for today’s applications. You have “consumers” as well as “providers” of data. The “middleware” glues these two worlds together and makes it possible to add basic services as the data spans both worlds.

Accessing Table data from the cloud

This diagram depicts how we we implement our system. Notice the “Solution Explorer” for Visual Studio. It represents our project and code that is running the Azure Data Center (The Microsoft Cloud)

Storage options

Here is a breakdown of the options:

Blobs	File system in the cloud
Tables	Massively scalable structured storage (key / value pairs)
Queues	Reliable storage and delivery of messages
Drives	Durable NTFS volumes for Windows Azure applications
SQL Azure	A relational database hosted in the cloud that can hold up to 50GB (9/27/2011)

Which data store is appropriate for you?

Here is a summary of their uses:

Binary Large Object (BLOB) Service	The simplest way to store text or binary data with Windows Azure. Think about large videos, images, pdfs, zip files.
Table Service	Is better for large amounts of data that need additional structure, which works exceptionally well with applications that need to work with data in a very detailed manner via queries. This is the data store that I will demonstrate in this post. The Table service offers structured storage in the form of tables. The Table service API is a REST API for working with tables and the data that they contain. The API is a Windows-based framework that you can access from most languages that run under Windows.
Queue Service	Is for reliable, persistent messaging between Web and Worker role instances.
Windows Azure Drive	Allows Windows Azure applications to mount a Page Blob, which is a single volume NTFS VHD. This allows applications to upload/download VHDs via blob

Performance Tips for Azure tables

Click here to get Azure performance tips

Additional resources for storage

Description	Link
Summary of Table Service Functionality	https://msdn.microsoft.com/en-us/library/dd135720.aspx
Overview of Windows Azure Storage	https://www.microsoft.com/windowsazure/features/storage/
Table Service REST API	https://msdn.microsoft.com/en-us/library/dd179423.aspx
Security Resources for Windows Azure	https://msdn.microsoft.com/en-us/library/ff934690.aspx

Your account drives the URL of the various storage abstractions

You can see how the RESTful interfaces map to the specific storage types.

Our RESTful service will leverage Tables

Scalable Structured Storage	Store Tables with billions of entities and TBs of data. Provides flexible schema (NoSQL).
Data Model	A table is a set of entities (rows). An entity is a set of properties (columns)
Familiar and Easy to use API	OData Protocol. WCF Data Services - .NET classes and LINQ

RESTful data

Description	Link
Windows Azure Storage Services REST API Reference	https://msdn.microsoft.com/en-us/library/windowsazure/dd179355.aspx#
Table Service REST API	https://msdn.microsoft.com/en-us/library/windowsazure/dd179423.aspx#
Blob Service REST API	https://msdn.microsoft.com/en-us/library/windowsazure/dd135733.aspx#
Queue Service REST API	https://msdn.microsoft.com/en-us/library/windowsazure/dd179363.aspx#

Key Whitepapers

Here is an excellent link to read more about Windows Azure.

Important Link	Address
Windows Azure Platform Whitepaper Downloads	https://www.microsoft.com/windowsazure/whitepaperdownload/

Next steps

Step	Description
Part 4	Building a cloud-based, RESTful service
Part 5	Building an Android client that consumes the RESTful data