Back before I joined Microsoft a little over 2 years ago, I began working on a project at my university on High-Performance Computing (HPC). We didn’t get too far as this really was more of a few year project and I left the Ph.D. program to join Microsoft. In any case, at the PDC this week there was a session on the new Windows 2003 Server Compute Cluster Edition. The demo around this shows using the standard Message Passing Interface (MPI) and the new job scheduler which allows you to specify the number of machines for the job to run on. As an added bonus, the Compute Cluster Edition plays nicely in Active Directory, so when you submit a job with a specified process, it gets run on all machines requested using the credentials of the account that submitted it. From the new administrator tools in the Compute Cluster Edition it seems you can also control various aspects of the machine pool.
Personally, when I get back to Redmond I’m going to try this out on a small number of machines.
UPDATE: The beta is available from http://connect.microsoft.com. When you get there, click on available programs in the left pane, log in with your Passport, and choose “Microsoft Cluster Compute Solution Beta”. You’ll be asked to fill out a form and then given access to the bits.
Also, BTW, the Computer Cluster edition requires and x64 processors…