Create Workitems for Test Failures in TeamBuild


Currently, TeamBuild does not have out of the box support to create workitems for test failures. I’ve attempted put some details below since this is a common request for several of our customers. It might look complicated if you are going to skim through this blog. But, I promise it is fairly straight forward 🙂  Read on…


 


The targets that are executed in a full stack TeamBuild are defined at %ProgramFiles%\MSBuild\Microsoft\VisualStudio\v8.0\TeamBuild\Microsoft.TeamFoundation.Build.targets. The target RunTestWithConfiguration is responsible for running tests.


 


The definition of this target is as follows:


 


<Target Name=RunTestWithConfiguration >


 


    <TeamBuildMessage


          Tag=Configuration


          Condition= ‘$(IsDesktopBuild)’!=’true’


          Value=$(Flavor) />


 





 


    <!– Test task for the end-to-end build –>


    <TestToolsTask


          Condition= ‘$(IsDesktopBuild)’!=’true’ and ‘%(MetaDataFile.Identity)’!=”


          BuildFlavor=$(Flavor)


          Platform=$(Platform)


          PublishServer=$(TeamFoundationServerUrl)


          PublishBuild=$(BuildNumber)


          SearchPathRoot=$(SearchPathRoot)


          PathToResultsFilesRoot=$(TestResultsRoot)


          MetaDataFile=%(MetaDataFile.Identity)


          RunConfigFile=$(RunConfigFile)


          TestLists=%(MetaDataFile.TestList)


          TeamProject=$(TeamProject)


  ContinueOnError=true />


 


    <!– Test task for the desktop build –>


    <TestToolsTask


          Condition= ‘$(IsDesktopBuild)’!=’false’ and ‘%(MetaDataFile.Identity)’!=”


          SearchPathRoot=$(SearchPathRoot)


          PathToResultsFilesRoot=$(TestResultsRoot)


          MetaDataFile=%(MetaDataFile.Identity)


          RunConfigFile=$(RunConfigFile)


          TestLists=%(MetaDataFile.TestList)


          ContinueOnError=true />


 


  </Target>


 


If you notice the highlighted part, ContinueOnError is set to true for the task TestToolsTask to ignore the test failures and continue the build process. We need to replace this with a new target and call it where there is an error in the target. MSBuild supports an element called OnError to achieve this behavior. Here is the new code after ContinueOnError is removed and OnError is added:


 


<Target Name=RunTestWithConfiguration >


 


    <TeamBuildMessage


          Tag=Configuration


          Condition= ‘$(IsDesktopBuild)’!=’true’


          Value=$(Flavor) />


 





 


    <!– Test task for the end-to-end build –>


    <TestToolsTask


          Condition= ‘$(IsDesktopBuild)’!=’true’ and ‘%(MetaDataFile.Identity)’!=”


          BuildFlavor=$(Flavor)


          Platform=$(Platform)


          PublishServer=$(TeamFoundationServerUrl)


          PublishBuild=$(BuildNumber)


          SearchPathRoot=$(SearchPathRoot)


          PathToResultsFilesRoot=$(TestResultsRoot)


          MetaDataFile=%(MetaDataFile.Identity)


          RunConfigFile=$(RunConfigFile)


          TestLists=%(MetaDataFile.TestList)


          TeamProject=$(TeamProject)


 />


 


    <!– Test task for the desktop build –>


    <TestToolsTask


          Condition= ‘$(IsDesktopBuild)’!=’false’ and ‘%(MetaDataFile.Identity)’!=”


          SearchPathRoot=$(SearchPathRoot)


          PathToResultsFilesRoot=$(TestResultsRoot)


          MetaDataFile=%(MetaDataFile.Identity)


          RunConfigFile=$(RunConfigFile)


          TestLists=%(MetaDataFile.TestList)


          ContinueOnError=true />


 


  <OnError ExecuteTargets=OnTestFailure; />


  </Target>


 


Note that I haven’t removed ContinueOnError for test task that runs for desktop build, because I don’t want to create bugs for test failures that happen in desktop build.


 


Now, we need to define OnTestFailure target. There is a task called CreateNewWorkitem available in TeamBuild to create workitems. I’ve used that here. Here is the target definition:


 


<Target Name=OnTestFailure>


   


    <CreateProperty Value=Work Item created by Team Build on BVT failure. >


      <Output TaskParameter=Value PropertyName=DescriptionText />


    </CreateProperty>


 


    <CreateProperty Value=Build log is at


        &lt;a


        href=’file:///$(DropLocation)\$(BuildNumber)\BuildLog.txt’


        &gt;


        $(DropLocation)\$(BuildNumber)\BuildLog.txt


        &lt;/a &gt;.


      >


      <Output TaskParameter=Value PropertyName=BuildlogFile />


    </CreateProperty>


 


    <CreateProperty Value=Test Results are at


        &lt;a


        href=’file:///$(DropLocation)\$(BuildNumber)\TestResults’


        &gt;


        $(DropLocation)\$(BuildNumber)\TestResults


        &lt;/a &gt;.


      >


      <Output TaskParameter=Value


              PropertyName=TestResultsFolder


              Condition=Exists(‘$(DropLocation)\$(BuildNumber)\TestResults’) />


    </CreateProperty>


   


    <CreateNewWorkItem


          BuildId=$(BuildNumber)


          Description=Tests are failed. $(BuildlogFile) $(TestResultsFolder)


          TeamProject=$(TeamProject)


          TeamFoundationServerUrl=$(TeamFoundationServerUrl)


          Title=BVT failures in build $(BuildNumber)


          WorkItemFieldValues=$(WorkItemFieldValues)


          WorkItemType=$(WorkItemType)


          ContinueOnError=true />


   


  </Target>


 


I’m creating a new workitem with title as “BVT failures in <build number>” and adding information about the build log file and test results folder in the description. I’m using the same workitem fields specified for build failures. You can find this property in TfsBuild.proj.


 


Now, I need to bring all these things together. To achieve this, I copied the target RunTestWithConfiguration from Microsoft.TeamFoundation.Build.targets and added to TfsBuild.proj, so that this version of the target is called in the build process. Now, I made the changes described above to this newly added target. After this, I added the target OnTestFailure as defined above. This is how it looked with all these changes:


 


<!–


I’ve pulled the target RunTestWithConfiguration (this is responsible for running tests) from


Microsoft.TeamFoundation.Build.targets into TfsBuild.proj and hooked in a new target called


OnTestFailure to create a workitem when there is a test failure. The changes I made are highlighted in bold.


  –>


 


  <Target Name=RunTestWithConfiguration >


 


    <TeamBuildMessage


          Tag=Configuration


          Condition= ‘$(IsDesktopBuild)’!=’true’


          Value=$(Flavor) />


 


    <TeamBuildMessage


          Tag=Platform


          Condition= ‘$(IsDesktopBuild)’!=’true’


          Value=$(Platform) />


 


    <!– SearchPathRoot for not Any CPU –>


    <CreateProperty


          Condition= ‘$(Platform)’!=’Any CPU’


          Value=$(BinariesRoot)\$(Platform)\$(Flavor)\ >


      <Output TaskParameter=Value PropertyName=SearchPathRoot />


    </CreateProperty>


 


    <!– SearchPathRoot for Any CPU –>


    <CreateProperty


          Condition= ‘$(Platform)’==’Any CPU’


          Value=$(BinariesRoot)\$(Flavor)\ >


      <Output TaskParameter=Value PropertyName=SearchPathRoot />


    </CreateProperty>


 


    <!– Test task for the end-to-end build –>


    <!– New: Remove ContinueOnError=true so that OnError target defined below can be called when there is a failure –>


    <TestToolsTask


          Condition= ‘$(IsDesktopBuild)’!=’true’ and ‘%(MetaDataFile.Identity)’!=”


          BuildFlavor=$(Flavor)


          Platform=$(Platform)


          PublishServer=$(TeamFoundationServerUrl)


          PublishBuild=$(BuildNumber)


          SearchPathRoot=$(SearchPathRoot)


          PathToResultsFilesRoot=$(TestResultsRoot)


          MetaDataFile=%(MetaDataFile.Identity)


          RunConfigFile=$(RunConfigFile)


          TestLists=%(MetaDataFile.TestList)


          TeamProject=$(TeamProject)


          />


 


    <!– Test task for the desktop build –>


    <TestToolsTask


          Condition= ‘$(IsDesktopBuild)’!=’false’ and ‘%(MetaDataFile.Identity)’!=”


          SearchPathRoot=$(SearchPathRoot)


          PathToResultsFilesRoot=$(TestResultsRoot)


          MetaDataFile=%(MetaDataFile.Identity)


          RunConfigFile=$(RunConfigFile)


          TestLists=%(MetaDataFile.TestList)


          ContinueOnError=true />


 


    <!–Execute OnTestFailure target when there is a test failure from end-to-end build –>


   


    <OnError ExecuteTargets=OnTestFailure; />


  </Target>


 


  <!–


    New: OnTestFauilre target definition


  –>


  <Target Name=OnTestFailure>


   


    <CreateProperty Value=Work Item created by Team Build on BVT failure. >


      <Output TaskParameter=Value PropertyName=DescriptionText />


    </CreateProperty>


 


    <CreateProperty Value=Build log is at


        &lt;a


        href=’file:///$(DropLocation)\$(BuildNumber)\BuildLog.txt’


        &gt;


        $(DropLocation)\$(BuildNumber)\BuildLog.txt


        &lt;/a &gt;.


      >


      <Output TaskParameter=Value PropertyName=BuildlogFile />


    </CreateProperty>


 


    <CreateProperty Value=Test Results are at


        &lt;a


        href=’file:///$(DropLocation)\$(BuildNumber)\TestResults’


        &gt;


        $(DropLocation)\$(BuildNumber)\TestResults


        &lt;/a &gt;.


      >


      <Output TaskParameter=Value


              PropertyName=TestResultsFolder


              Condition=Exists(‘$(DropLocation)\$(BuildNumber)\TestResults’) />


    </CreateProperty>


   


    <CreateNewWorkItem


          BuildId=$(BuildNumber)


          Description=Tests are failed. $(BuildlogFile) $(TestResultsFolder)


          TeamProject=$(TeamProject)


          TeamFoundationServerUrl=$(TeamFoundationServerUrl)


          Title=BVT failure in build $(BuildNumber)


          WorkItemFieldValues=$(WorkItemFieldValues)


          WorkItemType=$(WorkItemType)


          ContinueOnError=true />


   


  </Target>


 


Comments (0)

Skip to main content