The Real Meaning of Cloud Architecture

  • Article

How's that for a catchy title?

In the early days of the cloud, most people thought of it as a way to host applications: it was an IaaS (Infrastructure as a Service) world. I liked to call IaaS the "xcopy" model since it essentially means you take a VM and copy it into the cloud (with some network configuration, of course). It's a fairly simplistic model but, obviously, of great utility since the cost savings for an enterprise IT department can be quite significant. With large numbers of applications migrated as IaaS workloads IT departments can reduce or eliminate their dependence on their own data centers with consequent savings in real estate, hardware, electricity, air conditioning, and the like. It's a great story and it's probably still the predominant model of cloud computing today.

But in many ways the IaaS model is only the first step of cloud usage. Let me give you an illustrative example of what I mean. I'll use an IoT scenario but it's evocative, hopefully, of many other types of cloud applications. In this case the customer had deployed an application that consisted of a VM running in the cloud, some devices, and some logging, which was then analyzed and the results displayed on a custom dashboard. Sort of like this (please forgive my terrible drawing skills!):

               

That's all well and good. The application worked and was demonstratable.

But then I asked some questions.

  • What happens when you're really successful and you're monitoring perhaps thousands or tens of thousands of devices?
  • How do you detect patterns of usage? Patterns of failures?
  • How can you predict component failures or do predictive maintenance?
  • How can you watch for potential security breaches, or fake traffic?
  • In short, how can you write all the software you need to do this, quickly and reliably?

With many of the new services available today in Microsoft Azure, I suggested, you can do all of these things, and a lot more, today, with very little coding. Check out the diagram below:

 

 

So the devices send events (in REST or AMQP) which flow into Azure Event Hubs, which today process in aggregate billions (literally) of events per day. Azure Event Hubs provide a scalable way to do massive event ingestion; you only pay, however, for the bandwidth you need and use. In this case the events are picked up by Azure Stream Analytics which can do intelligent filtering and correlation. The data can then be placed in Azure Storage of one form or another which Big Data technologies such as Azure HDInsight (Hadoop) can analyze it and where data scientists using Azure Machine Learning can build experiments, train models, and quickly build applications to do predictions. PowerBI provides a visually pleasing user interface -- a dashboard --  which can be directly interfaced to Stream Analytics or a number of data sources including Azure SQL. Azure Service Bus provides a simple way to send queued or publish-and-subscribe messages and commands back to the devices. If you like, small little objects -- actors -- using Project Orleans today, the microservice-based Microsoft Service Fabric tomorrow, can implement server side logic for each individual device. And there are lots of other components that can be used to build even richer applications. And of course with the Azure Portal and Azure Application Insights your DevOps can monitor and manage your application.

Here's the key message: all this required very little code. Rather, the application -- and so many like it in Microsoft Azure -- can be wired together quickly. Comprised of services readily available in Azure, all fully tested in a wide variety of use cases and configurations, such applications reduce the need for to write custom code. (Prefer the open source versions -- say, Apache Storm in place of Stream Analytics? No problem!).

To me this sort of architecture represents the real value of cloud computing, that is, the ability to quickly wire together these high-value components with a minimum of coding (in some cases, none at all). So many IT professionals I talk to think of the cloud as purely a hosting opportunity but in fact the services available make massively scalable, feature rich applications easily accessible to enterprise IT.

Building a breakthrough application? I'd love to hear about it!