Don’t trust the weatherman? Running my own weather forecast on Windows HPC Server (Rain is coming back to Seattle Monday)

One of the interesting things living in Boulder Colorado for almost 20 years is that I bump into NCAR (National Center for Atmospheric Research), NOAA(National Oceanic and Atmospheric Administration), and CU Boulder researchers on a daily basis. I have asked people numerous times about how the weather forecast software works and they’ve always told me it’s complicated to run.

Well, not anymore. Partnering with a former Intel engineer, and the mighty power of my now 3 years old Cray cx1, we were able to download data from NCAR, run the WRF simulation code, and plot out weather for tomorrow (5/2, 5/3 of 2011).

Of course, it’s not without issues. We’ve been working on it for at least a month and a half by now in our spare time. Many of these software packages are designed to run on Unix and have been around for the last 20+ years. Porting the simulation software, posting the processing software onto Windows, and Ironing out all the bugs was a two-year effort that involved several other volunteers. Thank you!

I would like to spare you from having to read a thick book worth of details and show you the results instead. Basically, we used WRF, WPS (post processing), wget (ftp), NCAR Graphics, and a shell script to connect everything together. We’ve simplified it down to a command line tool with only 3 parameters: longitude, latitude and number of days. The rest has been automated (duct taped).

forecast –longitude –latitude –num_of_days

Downloading big data sets from noaa.

Computing the forecast!

Submitting the simulation part of the job.

The on premise nodes that got allocated get BUSY.  WRF sure does scale well on a cluster.

NCAR graphics post processing on precipitation.

Rain is coming, we’ll get some on Monday ……

More rain Tuesday….

Compute on a single Node varies from 1.5 hours to 3 hours.  I didn’t measure my multi-node runs yet. 

Thanks for reading my blog, and we’ll continue to report our progress of making this into a service / package for you to try.