Windows HPC 2008 demystified
First, let me explain what Windows HPC is all about. Windows High performance computing team has produced two versions thus far. Their first version is called Microsoft Windows Compute Cluster Server. It is a very basic solution to the traditional HPC space. It contains an MPI stack, and a job scheduler/ queuing system. These two packages sitting on top of Windows 2003 makes up the Windows Compute Cluster Server. Version one of the product is meant to support small clusters, typically 64 nodes or less.
For more information regarding HPC in general, please look my Problogue. As just reminder, HPC solutions usually involves using the Message Passing Interface Library (MPI). It is originally designed to solve large scale time/space complexity problems, where massively parallel machines each take a chunk of a giant matrix to solve. I have explained this in the previous blog series, Math under the hood, 5 min intro to Nonlinear solvers, explicit and implicit.
Since the release of version one, the team essentially re-engineered the code to bring the Job Scheduler/MPI stack to world class efficiency. This enables the product to meet or exceed the performance and scalability of competing platforms. The recent NCSA run using the windows HPC or Version 2 of the product, has landed Microsoft as #23 on the top 500 list. (68.48 TFlops and 77.7%) You'll find exactly the same hardware running Linux at 62.7 TFlops and 70% efficiency. There's no magic in all of this, the introduction of Network Direct allows MPI to bypass the traditional socket layer, and hook directly to the infiniband hardware. When you don't have to copy buffers through the various layers in an OS, it makes it feasible to achieve these outstanding results. For Version 2, the team has changed the name from Compute cluster Server to Windows HPC Server, because the V1 name "Compute Cluster Server" was often confused with a fail-over clustering solution. Of course, version 2 of the product has other significant improvements such as, the job scheduler and its new API, scheduler GUI, performance tools, diagnostic tools, deployment support, and much more. However, nothing stands out more than introduction of the new WCF Broker architecture, it allows Windows HPC Server to enable not just the scientists and engineers, but also businesses and industries to take advantage of the power of computing clusters.
In the new information age, a few milliseconds can often determine the success or failure of a business transaction. Having a computing cluster could dramatically improve the odds of success by giving the decision maker that few extra seconds he may need. For example, a lot of financial companies run pricing models for their bond portfolios, it is often in the form of an excel spreadsheet. I have personally witnessed some of the calculations requiring the user to take a coffee break and wait. If some of these calculations are encapsulated on the cluster, the computing time can be cut down to a mere few seconds. Technically, there's nothing complicated about the new model. Windows HPC simply provides you with integrated management, monitoring tools to run many instances of web services that provide computing services to your client applications.
I have put up a set of videos on Channel 9, the program manager Ming Xu shows you how easy it is to create such programs.
In this blog, we briefly covered windows HPC 2008. In the next few days, I'll put out more information, so that developers and system administrators can get their feet wet trying out the RC1 of the product.
Once again, thanks for reading my blog.