Node Reuse in MultiProc MSBuild


Greetings MSBuilders!


 


With the MultiProc support in MSBuild Orcas (now available in Visual Studio Orcas Beta 1), we’ve added the ability to reuse the MSBuild child nodes between builds.  When you’re doing multiple builds in a row, this helps reduce your total build time, by avoiding the start up costs of each child node.  By default, when doing a MultiProc build (using /m:2 or greater), nodeReuse is enabled.


 


Example:  You have dirs.proj (a traversal project) that builds p1.proj, p2.proj, and p3.proj.  Using the following command line to /m:3 dirs.proj


 


You will get 3 MSBuild nodes started, one parent node with 2 child nodes.  The 2 child nodes will, by default in this example, remain after your build completes.  If you were to build again (same projects, or different, with 2 or more nodes), the 2 child nodes will be re-used by that build.


 


Things to know with nodeReuse.  The timeout value for these child nodes is 60 seconds if they remain idle, before they will be terminated.  Because of these 60 second timeout, if you’re not doing multiple builds, you can disable nodeReuse (/nr:false) to avoid the extra processes from running after your build completes.  If a node was started by another user or the same user with different elevation access (Vista), those child nodes will not be reused.  You can only reuse a node that you created with the same level of access that they were started in.


 


[ Author: Eric, MSBuild Software Engineer ]

Comments (7)

  1. Todd Mancini on Configuring Visual Studio 2005 Team Foundation Server with Windows Sharepoint Services…

  2. A bit confused by the messaging, MSBuild is in the .NET Framework and not in Visual Studio – can you confirm that this is actually a feature of .NET Framework 3.5 rather than Visual Studio Orcas?

  3. Ashish says:

    It’s seems gd and also confusing. What I understand from the above blog is , In the above example dirs.proj depends on building three independent projects proj1,proj2 and proj3.

    Using the following command line  /m:3 dirs.proj :-> we create three threads of msbuild and build the three dependant projects simultaneously. This will help to reduce the overall time for Building.

    Am I thinking right or else correct it..

  4. Neil says:

    Stuart,

    MSBuild ships inside the .NET Framework Redistributable so all these features are part of .NET Framework 3.5. However, Visual Studio Orcas also includes the 3.5 redist, so if you install Orcas you’ll see this stuff there as well.

    Neil

  5. msbuild says:

    Ashish,

    Just by specifying /m:3 in the command line is not sufficient to enable building your projects in parallel. /m:3 just tells msbuild that it can use up to 3 nodes to build the projects. Nodes are basically different instances of the msbuild process and not threads within the same msbuild process (sub processes are started by the parent msbuild process).

    In order to enable parallel build of your projects you will, in additional, need to use the MSBuild task in your dirs.proj with BuildInParallel=true parameter and the project file names. This is what causes msbuild to build your projects in separate nodes. For instance if you use /m:3 and have 3 project files in your msbuild task then it will build the 3 project files in 3 nodes separate nodes (The parent msbuild also hosts a node) which will give you better performance on the overall build time.

    The node reuse aspect of this is that the nodes which have been started by the parent will automatically not go away (time out if 1 minute). Thus if you start a second build immediately – the same nodes will be reused saving us the process start up time.

    If this is still confusing please let us know and I will send you more information to clarify your concerns.

    Thanks,

    Jay Shrestha (jaysh@microsoft.com)

  6. gaddiel says:

    Nice concept. Great teamwork guys!

  7. I think that the node reuse aspect of this is that the nodes which have been started by the parent will automatically not go away. And if you start a second build – the same nodes will be reused saving us the process start up time.