Executing Unit Tests in parallel on a multi-CPU/core machine

In Visual Studio 2010, we introduced the ability to run tests in parallel.  Many machines today have multiple CPU’s or a CPU with multiple cores, so it makes sense that we may want our tests to run in such a way to use all the cores we have available on our machine.  This will effectively increase the number of tests running at the same time, which will reduce the time to run all the tests.


We looked at all the different test types and functionality that we have, and made the following decisions:

  1. The only test type that we would support for multi-CPU/core execution is the unit test.  The decision was based on various things such as: Is the test type doing UI testing? Is the test already doing some form of parallel execution? Is the test type hosted in a different host adapter (we may not have control over how the tests are executed)?  Are tests part of another test which may ensure a certain order of execution?  Is the test a unit test extension? (we may not have control over how the test is executed)? The answer to all these questions led us to support only unit tests.
  2. Data Adapters cannot be enabled.  Since data collection is based on test events, having data collection while multiple tests are execution quickly could slow things down.  Also, the data itself may not correctly be isolated to the test specified, but would include data or information from other tests that are executing.
  3. This only occurs for local execution through Visual Studio or MSTest.  Since remote execution allows you to already divide up tests across multiple agents, we have decided not to enable this scenario for remote execution.  Also, this currently is only available in Visual Studio or MSTest.


So, you ask, get on with telling me what I need to do!

Okay, but first we must remind you, and it also kind of goes without saying, but I will say it anyway; you need a machine with multiple cores.  Virtual machines can be used but you will need to make sure that the machine has multiple CPU’s.  An easy way to find out is to view the task manager.


My machine is single CPU, but has multiple cores.

Also, your tests must be thread-safe.  Only you can ensure that they are.  Failure to do so can result in incorrect results, deadlocks, and a lot of head-aches.  Take some time to review the thread-safe link and review your code for thread-safety.

How to: Enable parallel test execution

Okay, so now, how do we do it?  I will put all the steps from above into this list

  1. Ensure you have a multi-core/CPU machine (see above requirement).
  2. Ensure you are running only unit tests (see above decisions)
  3. Ensure your tests are thread-safe (see above requirement)
  4. Ensure you do not have any data adapters on (see above decisions)
  5. Ensure you are running locally (see above decisions)
  6. Modify your test settings file.
    Right-click the test setting file and select “Open With”
    1. Open as Xml
    2. Set the parallelTestCount attribute on the Execution element

      Options are:
      attribute not specified = 1 CPU/Core used (default)
      0 = Auto configure: We will use as many tests as we can based on your CPU and core count
      n = The number n of tests to run in parallel to use (if you do not want to use all of your CPU/cores)

    3. Save your settings… and you are done.


I have two cores, and so here is my resulting run:


As you can see, we have two tests running at the same time.  If we were running this in serial, we would at least take 10 seconds.  With parallel tests we got this down to 6 seconds


Of course there are many other factors that affect this number.  There is the cost of starting and tearing down the run, so you will see a lesser effect if you have a few tests.  It also depends on the number of CPU/cores you have and of course how fast your tests execute.

Good luck and hope this helps speed things up for you.

Bruce Taimana
Program Manager
Visual Studio Team Test

Comments (16)

  1. Jess Likens says:

    Excellent feature.  With 6+ cores being cheaply available now, this will provide massive performance gains.  We use this feature with our ecommerce project that has thousands of tests, and our CI build time has been cut by a factor of 5.  Of course, the caveat is that your tests follow more strict TDD guidelines, but generally speaking, unit tests should be independent.  There are some issues if you doing any threading magic in the code being tested (as Bruce alludes to when he says your tests must be thread safe), but other than that it is awesome to be able to take advantage of multi-core/CPU setups this way.

  2. Miguel Vazquez says:

    Great feature for teams that have independant unit tests like us. The best point is to let the runtime determine the number of cores on the running machine. It is going to help us speed up the validation process during a deployment.

  3. JT Keller says:

    Been trying to use this feature, but having problems.

      1) Does not work for a parallelTestCount > 5.  We get the error – "… number of hung tests exceeds maximum allowable '5'.

      2) Running tests in parallel in a single process is problematic.  These means your tests have to be thread safe.   This is not as easy to do as it sounds.  Any test that access an outside resource has problems.  For example, 2 tests that write to the same file (file access errors), or 2 tests that access and change a database – one test adds rows to a DB and the other test expects the DB to be empty.  Furthermore what happens when the code that you are testing is not thread-safe – yeah bad design maybe, but it may be acceptable in some cases. I know that you designed this feature to work with "simple unit tests", but in all reality these tests are a small portion on all tests that we write.  We use the test framework to write a multitude of system tests that we would like to run in parallel to dsvr us an enormous amount of time on a machine with multiple processors.  It would be better if this feature or one like it would run N processes and in each process run 1 test at a time.  Yes, there would still be problems with outside resources, but it would be more manageable because we could put all our conflicted tests in 1 file (tests are broken up accross the N processes on a per file/class basis).

  4. Vin says:

    This is neat.

    Is it possible to parallelize a Data-Driven unit test in Visual Studio 2010?

  5. Bruce Taimana says:

    JT Keller:  

    1. What is happening is that the cleanup thread is taking too long to complete.  After a test is complete, we kick off the cleanup, but, we don't wait around for a long time.  Should that clean up "hang" or take too long, we let it continue to cleanup but we kick off the next test in parallel.  If you reach 5 "hung" tests, we abort.

    2. I totally agree that getting to thread safety would be very difficult.  I also agree that basic unit tests are a small percentage of what is out there.  Management of the tests in the VS framework does not always guarantee that the tests in one test file will be executed in any particular order or executed together (though generally they do).  There are some underlying implementations that we had to work within to implement this feature, one of which being the indeterministic execution of tests.


    No, it is not possible for Data-Driven tests to be parallelized.

  6. Bryan Bell says:

    I'm quoting a reply by CoolDadTx from social.msdn.microsoft.com/…/206750bd-4ba2-42d2-a3cb-5f1f471c32fb since it was extremely helpful and took me forever to find after reading this blog post:


    It is working in RC.  You are not meeting one of the requirements.  Here is the checklist:

    1.Running a multi-processor machine.

    2.Can only have unit tests running.

    3.Data adapters must be disabled.

    4.Correctly modified the settings file to enable parallel execution using the parallelTestCount attribute.

    5.Running only local tests.

    Note that if you just create a new unit test project, write a couple of tests and then use F5 to run it then you won't get the parallel behavior.  You also might not see it if the tests execute too quickly.  

    There is no UI to set up the parallel tests.  If you make any changes to your test settings in the UI you'll lose the parallel settings.  Additionally (whether a bug or by design) the parallel test option is only read when the project is loaded.  Therefore if you change the setting you'll have to close the project and reopen it before it'll take effect.  I'm not sure if you can switch to another test setting and then back or not.  But that might work as well.

    Michael Taylor – 2/17/2010



  7. Bruce Taimana says:

    The part that was missing was here:

    Step 6: Additionally (whether a bug or by design) the parallel test option is only read when the project is loaded. Therefore if you change the setting you'll have to close the project and reopen it before it'll take effect.


  8. Alberto says:

    What about Visual Studio 2012? I cannot find these options.

  9. Uli Hüttinger says:

    You find them in the .testrunconfig file …

  10. Jaans says:

    Doesn't work with new VS2012 .runsettings?

    How do we run unit tests in parallel on VS2012?

  11. Venkata Sadineni says:


    I am trying to run data driven unit tests in parallel against Test controllerTest Agents. The tests are running fine. But as part of the tests we are logging to the Console.WriteLine.

    Some of the tests are throwing exceptions at the Console.WriteLine statements as below.

    My test code looks like: I have 5 tests similar to below one.


    [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "|DataDirectory|\TestData.xml", "Test1", DataAccessMethod.Sequential)]

    public void MyTest1()



    string val1 = testContext.DataRow["val1"].ToString();

    string val2 = testContext.DataRow["val2"].ToString();

    Console.WriteLine("val1: " + val1);

    Console.WriteLine("val2: " + val2);



    Error Message:

    Initialization method ParallelTestDataDriven.UnitTest1.TestInit threw exception. System.ObjectDisposedException: System.ObjectDisposedException: Cannot write to a closed TextWriter..

    Stack Trace:


    System.IO.StringWriter.Write(Char[] buffer, Int32 index, Int32 count)

    Microsoft.VisualStudio.TestTools.TestTypes.Unit.ThreadSafeStringWriter.Write(Char[] buffer, Int32 index, Int32 count)

    System.IO.TextWriter.WriteLine(String value)

    System.IO.TextWriter.SyncTextWriter.WriteLine(String value)

    ParallelTestDataDriven.UnitTest1.TestInit() in

    I tried to work around using TraceSource class. With this only logging is happening for the first row(in a multi row test). There is no logging from the 2nd data row onwards.

    I tried log4Net. With  this: the logging is getting jumbled. For eg: log for data row 2 is shown in data row 3.

    I am blocked on this. Any help is appreciated.

  12. Venkat Sadineni says:


    Can we run tests in parallel based on test categories?


    [TestCategory("1")] [TestMethod]

    public void Test1() {}

    [TestCategory("1")] [TestMethod]

    public void Test2() {}

    [TestCategory("3")] [TestMethod]

    public void Test3() {}

    I want to run all the tests which are having test category as "1" in parallel. Can you please let me know if there is a way.

  13. What about newer versions of Visual Studio??? says:

    Why was this feature removed in versions newer than 2010?

  14. Alex Smoilo says:

    It's a pity that the feature was removed from Visual Studio.

    I found a small tool ParallelTestRunner for Visual Studio on github which runs our Selenium tests in parallel from the command line.

  15. Good to know the capabilities. Thanks for sharing team!

  16. Albert says:

    Cool feature, looking forward to seeing more progress!