Hadoop for .NET Developers: Working with HDInsight on Azure GA

This last week, HDInsight on Azure became generally available.  This is great but with the roll out comes a few changes:


1. HTTPS communications with the Hadoop cluster are now on the standard HTTPS port, 443.  So instead of communicating over HTTP port 563, just identify your remote cluster using https://clustername or append a colon and port 443 to the end.

2. Some internal changes to HDInsight cause compatibility problems with older versions of the .NET SDK.  If you have any projects written against the HDInsight Preview version, please be sure to use NuGet to download the latest components or you may get either 404 errors or the cryptic “Can not create folder where file of same name already exists” error message.

3. HDInsight on Azure promotes the use of PowerShell for your interactions.  If you wish to use the Hadoop command line through a remote desktop session, navigate to the Configuration page on your cluster and enabled remote desktop connectivity there.


Hope that helps and happy Hadooping!

Comments (2)

  1. Ido Flatow says:

    I've updated the NuGet packages to the latest and now I get the following error:

    Could not find file Microsoft.WindowsAzure.Management.Framework.Threading.dll

    I understand that this is a bug with the new version of the NuGet.

    So we can't go forward, and we can't go back.

    Any ideas?

  2. I haven't experienced this issue. You might try updating the Windows Azure Storage components through NuGet to see if that does the trick.