"Virtual" HPC Cluster on a Notebook PC...?

  • Article

Maybe a stretch, eh?   Well maybe not if you just want to experiment with the tools and platform technologies.  Today, you can acquire a multi-core notebook supporting a CPU/RAM combination capable of running multiple virtual machines simultaneously.  This allows you to experiment not only with new Windows Server 2008 RC0 Virtualization (i.e. "Hyper-V"), but also Windows Compute Cluster Server and Visual Studio 2008 release candidate. 

A.  Objectives

1.  Construct a "virtual HPC cluster" using a laptop.

B.  What you'll need to acquire.

1.  Windows Server 2008 RC0 x64 (https://www.microsoft.com/windowsserver2008/audsel.mspx).  The Windows HPC Server beta1 release includes an RC0 x64 ISO.  It's also available via MSDN subscriber downloads.  Note that RC1 is coming soon.

2.  Windows HPC Server 2008 beta1 (https://connect.microsoft.com).   Be sure to download the SDK as well.

3.  A development environment.   The MS Platform SDK includes a x64 C++ compiler.  You'll want to download the SDK matching Windows Server 2008.   Use "Live.Com" to search "Server 2008 SDK".   Alternatively, download the Visual Studio 2008 release candidate.  Use "Live.Com" to search for "Visual Studio 2008".

4.  A nice multi-core laptop with a BIOS supporting "VT", i.e. "Virtualization Technology".  Enable this via the BIOS configuration.  You'll want at least 4 GB RAM, Be sure to confirm that the motherboard chip-set enables the OS (i.e. Windows Server 2008) to access all installed RAM.

5.  A capable external USB hard-disk drive to host the virtual machine files.  It's best to offload VM file I/O to a volume independent of your system volume.  Most USB disk drives are preformatted with the FAT32 file system format.  Be sure to reformat the external disk as an NTFS volume in order to better deal with the large VM files.

C.  Considerations.

1.  A dual-boot hard-drive configuration is ideal. 

2.  If your nice new laptop arrived with Vista pre-installed, you'll want to shrink the primary partition to create room for a second partition upon which to host Windows Server 2008.   Vista includes a new "diskpart" utility option named "shrink".  You can determine how much space is available to shrink using the "shrink querymax" command.  However, there are probably system files (e.g. paging file, system hybernation file) that occupy the upper segments of your drive and these will limit the "shrink" potential.   You may need to temporarily disable the virtual memory file ("My Computer, Properties, ...) and also disable hybernation (type "powercfg –h off").   Then, defrag the drive until you get about 25 GB available for your Server 2008 partition.   It's possible that you'll need to acquire an ubber-defrag utility like Rasco's "Perfect Disk 8" in order to move those stubborn system files

D.  Cluster Configuration

1.  First, install Windows Server 2008 RCx.   Although RC0 has been available since September, 2007, you may want to wait a bit for RC1  (it should be available soon).

2.  Enable the Server "Virtualization" role (https://www.microsoft.com/windowsserver2008/virtualization/install.mspx).

3.  First, add three new "Virtual Switches".  Choose the "internal" mode for each of these.  This just means they communicate local to the host and not externally via hardware network connections.  Label these "Public", "Management", and "MPI" networks.

3.  Add a VM...   This is easy using the Server 2008 x64 ISO image mounted as a CD/DVD and using the "New Machine" wizard.

4.  Use this first VM as the head-node.  

5.  Be sure to configure all base services (e.g. network connections, domain configuration) before installing the Compute Cluster Pack (CCP).  I recommend using only "internal" virtual network connections as this enables you to run AD/DNS/DHCP all within a virtual cluster with no external service dependencies.  Configure static IP addresses (e.g. 192.168.xx.1) on each head-node network connection.  Install AD on your headnode using "dcpromo.exe" and choose a nonsense domain name (e.g. "test.net").  DNS will install along with AD.  DHCP services will install with the CCP.

6.  Server 2008 Virtualization MMC includes a default console to each VM via the connect menu.  This is primarily for VM access during OS install or boot sequences.  After initial VM configuration you'll want to instead use "mstsc /console" from your host OS as this provides a richer full-screen remote desktop experience.

7.  Install the Compute Cluster Pack "Client Utilites" on your host OS.  This enables you to connect to the head-node virtual machine independently and deal with the cluster as if it were actually remote.  It's also useful to configure the host with development tools and the Compute Cluster Pack SDK.

E.  Reference Info

1.  Windows HPC Community web site:  https://WindowsHPC.net

2.  Windows HPC Product Portal:  https://www.microsoft.com/hpc

3.  Windows Virtualization:  https://www.microsoft.com/windowsserver2008/virtualization/default.mspx