Hello Azure Data Lake and U-SQL fans and followers. It has been a while since we release the release notes for all the cool features we released over the summer and listing all the pending deprecation items and breaking changes. The summer break is finally over, so without further ado, here are the Summer 2017…
Managing Pipeline & Recurring Jobs in Azure Data Lake Analytics Made Easy
Azure Data Lake Analytics (ADLA) is a big data job service that enables you to develop and run data transformation and processing jobs over petabytes of data. With no clusters or servers to provision or manage, you can process data on demand and scale instantly all while paying only for the jobs you run. This…
Directly store streaming data into Azure Data Lake with Azure Event Hubs Capture Provider
Azure Data Lake (ADL) customers use Azure Event Hubs extensively for ingesting streaming data – but up to now it was difficult for them to store raw and unprocessed events for an extended period for later batch processing. With the new Capture capability in Azure Event Hubs, you can easily store raw events into Azure…
U-SQL Tip: Generating ranges of numbers and dates
Many common scenarios for U-SQL developers require constructing a RowSet made up of a simple range of numbers or dates, for example the integers from 1 to 10. In this blog post we’ll take a look at options for doing this in U-SQL. In the process, we’ll get a chance to learn how to use some common U-SQL features:…
Running XGBoost on Azure HDInsight
XGBoost is a popular open-source distributed gradient boosting library used by many companies in production. Azure HDInsight is a fully managed Hadoop and Spark solution where you can easily create a fully-managed Spark cluster and with great extensibility. In this blog post, we will walk you through the detailed steps on how to compile and run XGBoost…
Create U-SQL EXTRACT Script Automatically
In this blog, you will learn how to create U-SQL EXTRACT script automatically using the latest version of Azure Data Lake Tools for Visual Studio. Watching this 3 minutes video to learn more. One of U-SQL’s core capabilities is to be able to schematize unstructured data on the fly without having to create a…
Integrating Kyligence Analytics Platform with Microsoft Azure HDInsight
This is a guest blog from Shaofeng Shi, Senior Architect from Kyligence Inc. Introducing Kyligence Analytics Platform Kyligence Analytics Platform (KAP) is an enterprise-ready big data warehouse on Apache Hadoop. Created by the same development team of Apache Kylin, an open-source distributed OLAP engine for big data, KAP inherits all Kylin’s advantages and has more innovations,…
Analyze data in Azure Data Lake Store using familiar-and-powerful Excel 2016
We are excited to announce that as part of the June 2017 updates of Excel 2016, Azure Data Lake Store is now supported as a source of data. Sophisticated and powerful tools like Excel and Power BI are preferred by many Enterprise data analysts to access and analyze data. As enterprises are building cloud-based data…
Azure Data Lake Tools for Visual Studio Code (VSCode) July updates
We are pleased to announce the July updates of Azure Data Lake Tools for VSCode. This is a quality milestone and we added local debug capability for C# code behind for window users, refined…
Solving the problem of “Problem with the SSL CA cert (path? access rights?)” for R server on HDInsight
R Server on HDInsight is an ideal platform for performing big data analysis using R interface. If you install packages just from the CRAN R repository, you probably won’t meet the problem mentioned in the title. However, if you want to install some package that is still under development, or if you want to use…