Hadoop for .NET Developers: Setting Up a Desktop Development Environment
NOTE This post is one in a series on Hadoop for .NET Developers.
If you are a .NET developer, you will want to setup a desktop development environment with the following components:
- Visual Studio 2010 or 2012
- NuGet Package Installer for Visual Studio
- A Local, Single Node Hadoop “Cluster”
Having these components installed on your desktop will allow you to develop against Hadoop locally as well as against a remote cluster (whether on-premise on in the cloud). You might be able to get away with not installing Hadoop locally, but most of the .NET-oriented documentation I’ve found assumes this is your setup.
I will assume you are comfortable installing Visual Studio on your own and the NuGet site provides simple enough installation options. I do recommend installing Visual Studio and NuGet first and then making sure your system is up-to-date with patches before proceeding with the Hadoop installation.
The Hadoop installation is very straightforward. That said, I once experienced an installation failure that prevented subsequent re-installation attempts. To fix this, I navigated to the <System>:Users<user name>AppDataLocalMicrosoftWeb Platform InstallerInstallers folder and deleted the installer associated with the Hadoop installation. After that, I was able to successfully re-install Hadoop.
Finally, I have found installing these components to a virtual machine running on my Windows 8 desktop (using Hyper-V) to provide me a reliable environment from within which to work and keeps my base desktop environment clean and stable. Whether you install to a VM or to your base desktop, please note that if your IP address changes, you will need to restart Hadoop as some components tend to hold onto the IP address with which the cluster was started. While I am told you could selectively restart components to clear the old IP address, I do not know which ones and have found that simply rebooting my development environment does the trick quickly enough :-)