We are excited to announce that Azure Stream Analytics can output to Azure Data Lake Store, a hyper-scale repository for big data analytics workloads.
This integration further advances the ease of enablement of a Lambda architecture where the same data that is subjected to real time stream analytics is also stored and then subjected to offline batch processing to unlock powerful insights. A large number of batch processing possibilities are enabled by Data Lake Store’s integration with Azure Data Lake Analytics, Azure HDInsight, upcoming integrations with Microsoft Revolution-R Enterprise, and Hadoop distributions from various industry-leading providers.
Azure Data Lake Store is built for supporting the storage needs of big data analytics systems that require massive throughput to query and analyze petabytes of data. It will be highly useful as data to be analyzed continues to grow exponentially, especially in streaming scenarios such as IoT.
The ability to output to Azure Data Lake Store from Stream Analytics can be enabled by using the respective Stream Analytics output choice as shown below:
When Data Lake Store is selected as an output in the Azure Management portal, it is necessary to authorize the usage of an existing Data Lake Store. Details on authorizing and configuring Data Lake Store can be found here.
At this time, the creation and configuration of Data Lake Store outputs is supported only in the Azure Classic Portal.
– Sam Chandrashekar, Program Manager