If you set out to install the Silverlight 1.1 Alpha SDK and installed all the additional recommended (but not required) bits such as Visual Studio Orcas and ASP.NET Future, you won't see this problem so you can stop reading now.
If you're a minimalist like me and install only the bare minimum Silverlight 1.1 Alpha runtime and SDK on a freshly constructed XP machine, you may discover that following the installation instructions for the Silverlight 1.1 Alpha SDK instructions won't quite produce a working set of quick start sample apps running on your localhost IIS server.
If after setting up the Silverlight 1.1 Alpha SDK QuickStart as instructed in the readme you fire up http://localhost/devpreview/ and get an ASP.NET exception message reporting that assembly System.Web.Extension cannot be found, don't bother scanning your hard disk for it. It's not on your machine.
To get it, you need to install the ASP.NET 2.0 AJAX 1.0 Web Extensions. After that, Silverlight 1.1 Alpha SDK examples run great on localhost, without all that Orcas stuff. Orcas is cool and all, but to get to know the Silverlight runtime up close and personal I like to examine it in isolation. It's good to know the difference between magic performed by the runtime itself and magic performed by the design time tools.
Oh, and if you happened to install the Silverlight 1.0 Beta runtime a few weeks ago to view videos when it was first rolled out, be sure to uninstall it before installing the Silverlight 1.1 Alpha. They don't appear to live side by side, but the 1.1 Alpha has all the 1.0 Beta runtime bits within it so you shouldn't miss anything by uninstalling the 1.0 Beta.
All clear now? It's ok. This is par for the course for an "alpha" release.