Ordered Tests in TFS Build

In an earlier article I discussed how to use and Ordered Test to control the execution order of Coded UI Tests (the same can be applied to other test types as well).  I received a few follow-up questions about how to do this in TFS Build so tests run in a particular order as part of a build.

Here’s one way that’s remarkably easy.

In my example, I have a project called JustTesting, which contains just a test project with 3 unit tests (which will always pass, BTW).

image

I put those tests into an ordered test:

image

In Solution Items, I open up my JustTesting.vsmdi file, create a new test list (called Ordered Tests), and add my ordered test to it.

image

Once that’s done, I check everything into TFS (my Team Project’s name is “Sample CMMI”.

Next, I set up a build definition (in Team Explorer, right-click Builds, and select “New Build Definition”).  Set whatever options you want (name, trigger, workspace, build defaults) but stop at “Process”.

In the section named “2. Basic”, you’ll see that by default the Automated Tests field is set to (something like): “Run tests in assemblies matching **\*test*.dll using settings from $/Sample CMMI/JustTesting/Local.testsettings”. 

image

Click on the ellipsis on the right of that to open the Automated Tests dialog:

image

Remove the entry you see (or leave it if you wish to include that test definition), and then click “Add”.

In the Add/Edit Test dialog, select the optoin for “Test metadata file (.vsmdi)”.  Use the browse button to find and select your desired .vsmdi file.  In my example, JustTesting.vsmdi.

Uncheck “Run all tests in this VSMDI file”, then check the box next to your test list containing the ordered test.  In my example, the test list is called “Ordered Tests”.  Your dialog should look something like this:

image

Click OK and you’re Automated Tests dialog should look like:

image

Click OK again, then save your build definition.

Queue a new build using this definition.  Once complete, look at the build report to see your test results.

image

image

It’s a few steps, but nothing ridiculous.  And I didn’t have to hack any XML files or do any custom coding.

Hope this helps!