One of the most frequent questions I was asked at the booth at VSLive/SpeechTek was whether the Speech Application SDK required Whidbey/VisualStudio 2005. I think, this was largely because the other demos during Bill Gates' speech were of features new in Whidbey.
Understandably, people are concerned about developing for a new server platforming using tools that are still at a tech preview stage. However, just to clarify completely:
The Microsoft Speech Application SDK for Speech Server 2004 will run on Visual Studio 2003
You do not need VisualStudio 2005 (Whidbey) for the Speech Application SDK
In other words you can use it right now! You can download Beta 4 here, and we're expecting that the final version will be up in a few weeks.