Organize your pipeline and recurring jobs easily with Data Lake Analytics (part 2)

Identify performance problems and reduce failed jobs with pipeline and recurring job information In part 1, we showed you how to tag and explore your pipeline and recurring jobs: Tag jobs with the new pipeline and recurring job metadata using Azure PowerShell Explore pipeline and recurring jobs using Azure Portal and Azure PowerShell In this…

Organize your pipeline and recurring jobs easily with Data Lake Analytics (part 1)

 Tagging and exploring pipeline and recurring jobs with metadata In a previous blog post, we introduced how managing/monitoring pipeline and recurring jobs is made easier with new concepts we added to Data Lake Analytics. Let’s take a deeper look at how to leverage this new capability. This is part 1 of a 2-part series. In…

Azure Data Lake Analytics and U-SQL Summer 2017 Updates: Introducing GZip on OUTPUT, Catalog Views, Major updates to Cognitive Libraries, Tool support to create your EXTRACT statement and much more!

Hello Azure Data Lake and U-SQL fans and followers. It has been a while since we release the release notes for all the cool features we released over the summer and listing all the pending deprecation items and breaking changes. The summer break is finally over, so without further ado, here are the Summer 2017…


Azure Data Lake U-SQL April 25 2017 Updates: Introducing Packages, UNPIVOT INCLUDE NULLS, fast file set preview flag, R extension returns dataframes, exporting your cluster database with sample data to your local run and more!

We have concluded the rollout of our April 2017 refresh to all the regions today. Here are the April 2017 Updates for Azure Data Lake U-SQL and Developer Tooling! The main items are the release of the package feature that allows you to bundle assembly reference statements, variable declarations into a shareable package and reduce…


Azure Data Lake U-SQL March 9 2017 Updates: Deprecations turn into errors, PIVOT/UNPIVOT, cross ADLS account U-SQL catalog sharing, nuget packages and more!

After mainly internal service updates after our general availability, we released several new U-SQL features in our release last week. Note that these updates are now available in all regions, including the new Europe North region. Here are the March 9 2017 Updates for Azure Data Lake U-SQL and Developer Tooling! The main take away…


Azure Data Lake Tools for VSCode (Preview) – March Update

Continue our journey to launch Azure Data Lake Tools for VSCode for better cross-platform support, meet developers where they are in Mac, Linux and Windows, and deliver a first class light weight code editor experiences for U-SQL. We are pleased to announce our March release which includes a few important features.


Enabling U-SQL Advanced Analytics for Local Execution

After we announced the ability for U-SQL to massively distributed Python code in the Azure Data Lake Analytics service, a lot of developers have been asking us when the the Python support will work using U-SQL Local Execution. In this post, we’ll describe how to make that happen. To set expectations, what we will describe below is not officially supported – we do…


U-SQL Deprecation Update: Migration of Data Source Credentials and Removal of CREATE CREDENTIAL, ALTER CREDENTIAL and DROP CREDENTIAL

Back in October, we announced that we simplified the U-SQL Credentials by merging the password secrets that are being created in Powershell and the other parts of the credential object into credentials that are being completely created with a Powershell command. This reduces one statement from the creation process. During the initial phase, we did…


U-SQL Deprecation notice: PARTITION BY BUCKET will be removed

Hi all In the upcoming refresh, we are removing the deprecated syntax PARTITION BY BUCKET and will raise an error. Thus, if you have not yet updated your table definitions with the previously announced new syntax, please do so now or your scripts will fail starting some day in February! For more information, please see:…


Introducing: Microsoft Azure Data Lake Tools for Visual Studio Code

Welcome to the Microsoft Azure Data Lake Tools preview for Visual Studio Code, an extension for developing U-SQL projects against Microsoft Azure Data Lake! This extension provides you a cross-platform, light-weight, keyboard-focused authoring experience for U-SQL while maintaining a full set of development functions. Want to make this extension even more awesome? Share your feedback….