A Tale of Two Compilers


In previous posts, I have hinted at the fact that there is more than one C# compiler on a machine with Visual Studio and .NET Framework installed. Sometimes there are several.

Simply put, when we release Visual Studio we release a compiler referred to as the in-process compiler, or in-proc compiler. We generally also release a new version of the .NET Framework at the same time. In the .NET Framework we also ship a separate compiler: the framework compiler, CSC.EXE. The in-proc compiler is tucked away in a Visual Studio DLL containing a bunch of other code as well. The presence of multiple compilers can result in an awkward servicing story and general confusion.

Why two compilers? Well, that wasn’t the original plan, but late in the Visual Studio 2005 cycle, the plan changed. The reason that Visual Studio doesn’t just use the framework compiler is for performance. Using the in-proc compiler avoids the cost of spinning up another process, and it also reuses a database of interned strings. These issues may not seem significant today, but at one time they had a real performance impact.

The downside of using the in-proc compiler is that it limits scalability. The Visual Studio address space is pretty crowded, and compiling large projects takes up a lot of address space. This wouldn’t be a concern if Visual Studio spawned CSC.EXE for the build. So if you run into build scalability issues with a Visual Studio full build, you can often work around them by invoking MSBUILD.EXE from the command line supplying the .SLN file.

I hear you saying, “But my Visual Studio is spawning CSC.EXE. I see a message saying so in the output window.” When the output window of Visual Studio tells you the command-line it is invoking CSC.EXE with, don’t believe it. Visual Studio is calling the in-proc compiler with the equivalent switches, not the framework compiler. This may change for future versions of Visual Studio, but in Orcas, you’re getting the in-proc compiler.

The presence of two compilers per release can pose a problem for servicing. Visual Studio and .NET Framework generally have two different servicing schedules. This means that sometimes users may see a fix in Visual Studio but not in CSC.EXE or vice versa. For service packs to Visual Studio 2005 and .NET Framework 2.0 this is definitely the case. Several more bugs were fixed in CSC.EXE in .NET Framework 2.0 SP1 (and SP2) than were in Visual Studio 2005 SP1. Thankfully, servicing of .NET Framework 3.5 and Visual Studio 2008 is happening at almost the same time. Right now we’re working on .NET Framework 3.5 SP1 and Visual Studio 2008 SP1 and all fixes made so far have been made to both compilers.

Speaking of servicing, we really appreciate all of the defect reports we get through Connect. I’ve heard a lot of grumbling about Connect, but do know that the C# compiler team places very high value on reports coming through this forum.


Comments (5)

  1. int19h says:

    > Why two compilers? Well, that wasn’t the original plan, but late in the Visual Studio 2005 cycle, the plan changed. The reason that Visual Studio doesn’t just use the framework compiler is for performance. Using the in-proc compiler avoids the cost of spinning up another process, and it also reuses a database of interned strings. These issues may not seem significant today, but at one time they had a real performance impact.

    This is a good explanation for having a library, but why csc.exe isn’t just a console frontend to the same library, then?

  2. Weeble says:

    Does the refactoring engine count as a third compiler, then? Or is it one of the two existing ones running in a special mode? I was surprised when I discovered that code can compile normally, but generate compile errors in the refactoring engine. (Already reported through Connect. It was to do with how expressions in arguments to attributes are resolved.)

  3. kevinowen says:

    You can also make Visual Studio actually spawn CSC.EXE for builds inside the IDE by setting the UseHostCompilerIfAvailable property in your project (.csproj) file to false (it defaults to true). Just put the following in the first PropertyGroup element at the top of your project file:

    <UseHostCompilerIfAvailable>false</UseHostCompilerIfAvailable>

  4. Ed Maurer says:

    int19h, you’re right. I only told part of the story. It’s true that the compiler guts could easily be a DLL that is linked to by both CSC.EXE and VS, and that was the VS 2005 design. For legal reasons that I won’t get into, that was scuttled.

    Weeble, the refactoring engine in VS 2008 is a combination of the in-proc compiler run in a special mode to produce some internal data structures along with special purpose VS code to interpret them. The live semantic errors feature that is debuting in VS 2008 SP1 makes use of a path through the in-proc compiler code that is similar to that used by IntelliSense.

  5. Stewart Rogers says:

    I have been using VS2008 to build a solution with 36 projects on a high end laptop. The build times have been in the range of 75 sec. the development team started investigating Engineering Workstations with 64 bit quad core XEON processors from Lenovo. We put in RAM Disc for the solution directory and manage our source with TFS. Our first build times using the VS2008 IDE were on the order of 50 sec and the processors performance graph showed approximately 25% utilization across all four processors. After reading this blog and added a .BAT file to automate the build.

    "%SystemRoot%Microsoft.NETFrameworkv3.5msbuild.exe" "..XXXX.sln" /p:"Platform=Mixed Platforms" /p:Configuration=Debug /t:rebuild /maxcpucount:20

    Using the Lenovo Engineering Workstation the automated build took only 10 sec. to complete compared to the identical solution taking 50 sec in VS2008. I would have expected the same performance from the IDE; obviously the IDE’s complier could be improved. I would like to recommend the purchase of the Lenovo workstations but can’t unless the IDE can come close to the same performance as the MSBuild.exe provides. Can you give any guidance on IDE build performance on upcoming releases?