Cloudera clusters now run with Azure Data Lake Store

  • Article

We are excited to announce that with today's release of Cloudera Enterprise 5.11 you can now run Spark, Hive, and MapReduce workloads in a Cloudera cluster on Azure Data Lake Store (ADLS). Cloudera customers can now take advantage of the many benefits of running clusters on ADLS. And ADLS brings to its customers another valuable compute platform from which customers can gain insights on their big data.

With Cloudera Enterprise 5.11, Cloudera customers can now take full advantage of the benefits of separating compute and storage to expand and contract their compute resources and storage resources independently in response to the enterprise workload time and cost requirements. Cloudera customers can easily create transient clusters without having to move data. And multiple clusters can easily and concurrently access shared data in Azure Data Lake Store optimized for disparate workloads such as product design, adhoc analytics on raw data or customer insights. You also get high durability at lower cost with ADLS relative to attached storage where data is replicated twice, once at the HDFS layer and once at the storage layer.
cdh-adls-slide-2

The Azure Data Lake (ADL) vision from the beginning has been to transform business data into intelligence by providing analytics on any data at cloud scale. ADL enterprise customers gain insights on their business data using a wide range of tools and platforms. Today’s release of Cloudera Enterprise 5.11 brings another very valuable and widely-used Hadoop computation platform to the set of platforms that can leverage ADLS. No matter what big data analytics platform you choose, Azure Data Lake Store provides a single high throughput enterprise-scale hierarchical file system data lake repository for big data.

Anyone with an Azure subscription can now deploy Cloudera clusters with ADLS. To get started, you can use the Cloudera Enterprise Data Hub template or the Cloudera Director template on Azure Marketplace to create a Cloudera cluster. Once the cluster is up, see here for more information on how to set up your Cloudera cluster with ADLS today!