Microsoft Azure, Linux and enhanced data services HDInsight, Hadoop, DocumentDB and Mobile Engagement

Today Microsoft announced new and enhanced Microsoft data services: a preview of Azure HDInsight running on Linux, the general availability of Storm on HDInsight, the general availability of Azure Machine Learning, and the availability of Informatica technology on Azure.

Azure HDInsight Now Available on Linux

Microsoft  announced Azure HDInsight built on Linux, in addition to Windows, making it the first Microsoft managed service that uses Linux. This means if your running Hadoop on Linux on-premises, you can now leverage common Linux tools, documentation, and templates to extend your deployment to Azure Cloud. More information can be found on https://azure.com/hdinsight

Storm for Azure HDInsight Now Generally Available Making Real-Time Analytics Simple

Microsoft is making Apache Storm for HDInsight generally available, giving you  a simple way to deploy real-time capabilities on Hadoop in a few clicks. Storm is an open-source stream analytics platform that can process millions of “events” in real time. As a part of general availability, Microsoft is also making Storm available for both .NET and Java and the ability to develop, deploy, and debug real-time Storm applications directly in Visual Studio. More information can be found on https://go.microsoft.com/fwlink/?LinkID=525875&clcid=0x409.

Azure HDInsight Built on Hadoop 2.6

In conjunction with Hortonworks availability of HDP 2.2, Microsoft  HDInsight built on Hadoop 2.6, Hive and Pig 0.14, HBase 0.98.4 and more. Teaming up with Hortonworks and the open source community, this version of HDInsight includes work done on Stinger.next to speed up Hadoop queries with the goal of achieving sub-second response times. The first phase of Stinger.next is now in HDInsight running Hive 0.14.  Pig can now process data in ORC files, and can leverage Tez as an execution engine. More information can be found on https://azure.com/hdinsight.

Azure HDInsight now runs on more VM sizes to support Big Data

HDInsight can now utilize A2 to A7 sizes built for general purposes, D-Series nodes that feature solid-state drives (SSDs) and 60-percent faster processors, and A8 and A9 sizes that have InfiniBand support for fast networking. HBase for HDInsight customers can benefit from the higher memory from the D-Series to increase performance. Storm for HDInsight customers can also benefit from higher memory for loading larger reference data and faster CPU’s for higher throughput. Pricing details can be found https://azure.microsoft.com/en-us/pricing/details/hdinsight/.

Azure HDInsight now supports Cluster Scaling

HDInsight is also delivering the general availability of a highly requested feature, “Cluster Scaling” in Azure HDInsight.  With this feature, you will be able to easily change the number of nodes of a running HDInsight cluster without having to delete and recreate a new cluster.  This feature is available in the Azure Management Portal today.

Azure Document DB Hadoop Connector

The availability of the Hadoop Connector for DocumentDB allows users to perform complex analytics jobs on their data within the Apache Hadoop framework. DocumentDB databases can now function as data sources and sinks for Pig, Apache Hive, and MapReduce jobs. By passing queries to DocumentDB to take advantage of its rich querying capabilities, data can be reduced and filtered before Hadoop processing, helping increase the efficiency of data aggregations and manipulations. The Hadoop Connector is also compatible with Azure HDInsight. This enables a broad range of analytics scenarios over unstructured data in many industries, including gaming, energy, and healthcare. The Hadoop Connector is available now as open source software on GitHub and Maven.

Public Preview of Azure Mobile Engagement

With the Mobile Engagement public preview, organizations can now have access to real-time actionable analytics to increase app usage, open APIs to help ensure data can be leveraged from existing CRM, CMS and other business related systems, create targeted campaigns through intelligent customer segments and in-app messaging capabilities all while ensuring that their data and privacy remains protected. Organizations can easily and continually enhance and optimize the user experience – driving higher retention rates and increased app usage. More information can be found at the  Mobile Engagement webpage.