Incorporate “Big Data” Within Your Visual Studio App Using Azure HDInsight
To help make it easier to work with “big data” in your apps, we added support for Hive queries and management of data clusters with Azure HDInsight, the Microsoft Azure Hadoop cluster solution for Visual Studio. For Visual Studio 2012 and 2013, HDInsight support comes in a Visual Studio extension in the Azure SDK, and it’s baked into Visual Studio 2015 Preview. The extension makes it easier for you to visualize your Hadoop clusters, tables, and associated storage from within Visual Studio. It also supports creating and submitting ad hoc Hive queries for HDInsight directly against a cluster from within Visual Studio and building a Hive app that is managed like any other Visual Studio project.
Download the Azure Tools 2.5 SDK now for Visual Studio 2012 | Visual Studio 2013 | Visual Studio 2015 Preview.
You’ll find HDInsight support in the Visual Studio Server Explorer, alongside other cloud services under the Azure node. Open the node and you’ll find that it’s now simple to explore clusters and Hive tables and their schemas, right down to immediate queries of the first 100 rows of a table for quick preview. This immediacy of the UI helps you to quickly understand the shape of the data you are working with in Visual Studio.
You can also use this extension to create Hive queries and submit them as jobs. Use the context menu against a Hadoop cluster to immediately begin writing Hive query scripts. In the example below, we create a simple query against a Hive table with geographic info to find the count of all countries and sort them by country. The Job Browser tool helps you visualize the job submissions and status. Double click on any job to get a summary and details in the Hive Job Summary window.
You can also navigate to any Azure Blob container and open it to work with the files contained there. The backing store is associated with the Hadoop cluster during cluster creation in the Azure dashboard. However, you still manage the Hadoop cluster in the Azure dashboard.
For more complex script development and lifecycle management, you can create Hive projects within Visual Studio. In the new project dialog (see below) you will find a new HDInsight Template category. The Hive Sample project type is a helpful starting point since it is prepopulated with a more complex Hive query and sample data, targeting the common case of web log processing.
To get started visit the Azure HDInsight page to learn about Hadoop features on Azure. Check out the getting started guide for how to use HDInsight Hadoop Tools for Visual Studio and Channel 9 video on HDInsights Tools in Visual Studio.
To give us feedback or if you have questions, check out the MSDN forum or find us on stackoverflow.
 |
Oliver Chiu, Product Marketing, Hadoop/Big Data and Data Warehousing
Oliver Chiu is a Product Marketer focusing on Hadoop technologies like Azure HDInsight and Microsoft’s data warehousing suite. |
Light
Dark
0 comments