Check it out!
I've read about symbol servers and source servers before (in John Robbins' excellent Debugging Microsoft .NET 2.0 Applications book, for example, and also due to the recent news that Visual Studio 2008 will support source-code debugging of the .NET Framework Libraries), but never really got around to trying them out since they seemed pretty complex to set up. However, I've recently set up a source server on a Team Foundation build server (twice!), and my eyes have now gone wide open: anybody who references assemblies built by those TFS build servers can now seamlessly debug them.
When an assembly is "source server-enabled", the pdb file will contain the full path and the exact version of the file in source control that was used to build that assembly. If the debugger then enters a method, Visual Studio automatically downloads that correct file, places it in a local cache, and opens it for debugging. This is super sweet!
Now the hard work was already done and published by Roy Osherove, but there were some extra steps to take if you wanted to integrate this in a real Team Build (i.e. on the TFS build server instead of on a local machine) and it also lacked some additional information to get everything working. So I put together a full document that covers all aspects of setting up a source server, modifying the Team Build script, configuring the project files, and finally configuring the development machines.
The full document (PDF) can be downloaded here, and you can read the one-page summary that briefly covers the required steps below. I cannot recommend setting up a source server enough, it's free and takes less than an hour if you simply follow the procedure, and the advantages are - hopefully - obvious.