How to running multiple tests in team build?

  • Article

 

In beta3 you have better control on the way tests are executed. Some common scenarios are:-

  • To run tests of different categories:-
    • You need to modify tfsbuild.proj file to introduce additional test category. For example if your metadata item is defined as

<MetaDataFile Include = "$(SolutionRoot)\HelloWorld\HelloWorld.vsmdi">

  <TestList> BVT1 </TestList>

     </MetaDataFile>

and you want to execute another category of test (i.e. BVT2). You can easily add the additional test category.

<MetaDataFile Include="$(SolutionRoot)\HelloWorld\HelloWorld.vsmdi">

<TestList> BVT1; BVT2 </TestList>

</MetaDataFile>

    • Do note that build will fail if one of the following things hold true
      • Test category BVT2 is not defined in HelloWorld.vsmdi
      • BVT2 contain tests that evaluates to inconclusive
  • Using multiple vsmdi files to running tests:-
    • This is not working on beta3 bits. There is a bug in file Microsoft.TeamFoundation.Build.targets. You need to make the following change at two locations in above mentioned targets file.

From:-

<TestToolsTask

Condition = " '$(IsDesktopBuild)' ! = 'true' "

…

To:-

<TestToolsTask

Condition =" '$(IsDesktopBuild)' ! = 'true' and ' %(MetaDataFile.Identity)' ! = '' "

…

Do note that this is fixed in post beta3 sources.

    • If want to run additional tests defined in different vsmdi file using team build. You need to define new item for the corresponding vsmdi file. For example you want to run test of the category “BVT2” defined in HelloWorld2.vsmdi. You need to do the following steps:-

      • Modify tfsbuild.proj file and add new MetaDataFile item for HelloWorld2.vsmdi file

      <ItemGroup>

 

<MetaDataFile Include = "$(SolutionRoot)\HelloWorld\HelloWorld.vsmdi">

        <TestList> BVT1 </TestList>

</MetaDataFile>

 

<MetaDataFile Include = "$(SolutionRoot)\HelloWorld\HelloWorld2.vsmdi">

        <TestList> BVT2 </TestList>

</MetaDataFile>

 

      </ItemGroup>

  • Running tests selectively for particular “platform|configuration” combination:-

     

    • Make sure you have the above mentioned fix of “Microsoft.teamfoundation.build.targets” file.
    • Modify tfsbuild.proj file. For example if you want to run the test for only “Debug|Any CPU” configuration, you need to modify the corresponding MetaDataFile item and add the condition (evaluate this item value only when "platform|configuration" is "Any CPU|Debug")

<MetaDataFile

            Condition = ” ’$(Platform)’ = = ’Any CPU’ and ‘$(Flavor)’ = = ’Debug’ ”

Include = "$(SolutionRoot)\HelloWorld\HelloWorld.vsmdi">

      <TestList>BVT1</TestList>

</MetaDataFile>