Automate HDInsight Cluster Creation using .net sdk

This blog post shows how to automate HDInsight cluster creation from .net sdk. Information is available in pieces so I thought to blog about it so that I can refer it in future and it may help others.

This blog post shows how it can be done but when you execute program it will prompt for authentication which will not work when you want to automate the entire process.

We want to automate the process means no manual intervention is required to provide credential for authentication. In below steps I am setting up a application in Azure AD (Active Directory) which is a must if you want to automate the process, providing access to resource group and then .net code setting up HDInsight cluster.

In this scenario, I have test resource group created and a storage in it from Ibiza portal.

First step is to create Azure Active Directory application and service principal to access resources on Azure. Please follow here to setup.

Once Azure Active Directory application and role assignment  is configured it will look like below.

Once it’s done. Open Visual Studio 2015. Create new console application.

Add reference mentioned here.

Download code from here and paste it.

Execute the code.

Switch to Azure portal to monitor if request has been accepted.

After sometime you will get message that HDInsight cluster is created.

Hope it helps.

Eat Healthy, Stay Fit and Keep Learning!