Summary results file for VSTS Generic Tests

  • Article

So I am using Visual Studio Team System for prototyping some new Orcas features and am just loving it.  What I would to do today is share some code I used to generate a 'Summary results file'.

When a VSTS Generic Test has completed, VS looks for an optional XML file to display a test summary.  There is a schema for this XML file, but unfortunately no object model for you to use.

What I did was used a tool called XSD to generate an object model around that schema, then wrote my own result log wrapper. 

To generate a code object model around the schema, do the following:

  1. Open up the VS 2005 Command Prompt (from Start menu)
  2. CD ..\XML\Schemas
  3. XSD /classes /language:VB SummaryResult.xsd

That's it!  Now you have a class file representing that schema.  You can load and save XML files conforming to that schema using the XmlSerializer class.

The following is the code I used to wrap the SummaryResults object model; it doesn't do anything too fancy, just reports PASS/FAIL results:

Option Strict On
Option Explicit On

Imports System.IO
Imports System.Xml.Serialization

''' <summary>
''' Results log for tracking scenarios and PASS/FAIL results. Simply a
''' wrapper for SummaryResults which has been generated by XSD.
''' </summary>
''' <remarks></remarks>
Public Class ResultsLog

 ''' <summary>
''' This is the object model for the SummaryResults XML file.
''' </summary>
Private m_summaryResults As SummaryResult

 ''' <summary>
''' The current InnerTest which is executing.
''' </summary>
Private m_currentScenario As SummaryResultInnerTest

 ''' <summary>
''' A compiled list of completed InnerTests.
''' </summary>
Private m_scenarios As List(Of SummaryResultInnerTest)

 Public Sub New()
m_summaryResults = New SummaryResult()
m_summaryResults.TestName = "<testcase name not set>"

  m_scenarios = New List(Of SummaryResultInnerTest)

  m_currentScenario = New SummaryResultInnerTest
m_currentScenario.TestName = "Testcase Init"

  m_currentScenario.TestResult = testResultType.Passed
 End Sub

 ''' <summary>
''' Property describing the TestcaseName
''' </summary>
Public Property TestcaseName() As String
Get
Return m_summaryResults.TestName
End Get
Set(ByVal value As String)
m_summaryResults.TestName = value
End Set
End Property

 ''' <summary>
''' Starts a new test scenario (InnerTest)
''' </summary>
Public Sub InitScenario(ByVal scenarioName As String)
' Add the current scenario to our results file
m_scenarios.Add(m_currentScenario)

  ' Move onto the next scenario
m_currentScenario = New SummaryResultInnerTest
m_currentScenario.TestName = scenarioName
m_currentScenario.TestResult = testResultType.Passed ' Assume passed until a Failure is issued

 End Sub

 ''' <summary>
''' Saves the results
''' </summary>
''' <remarks></remarks>
Public Sub SaveResults(Optional ByVal resultFilename As String = "Results.xml")

  ' Add the current scenario to our results file
m_scenarios.Add(m_currentScenario)

  ' Add all scenarios to the test
m_summaryResults.InnerTests = m_scenarios.ToArray()

  ' Set final result
m_summaryResults.TestResult = testResultType.Passed
For Each scenario As SummaryResultInnerTest In m_scenarios
If scenario.TestResult <> testResultType.Passed Then
m_summaryResults.TestResult = testResultType.Failed
m_summaryResults.ErrorMessage = scenario.ErrorMessage
Exit For
End If
Next

  ' Save that puppy out
Dim resultStream As New StreamWriter(resultFilename, False)

  Dim serializer As New XmlSerializer(GetType(SummaryResult))
serializer.Serialize(resultStream, m_summaryResults)

  resultStream.Close()
End Sub

 ''' <summary>
''' Logs a failure in the current inner-test
''' </summary>
Public Sub LogFailure(ByVal message As String)
m_currentScenario.ErrorMessage = message
m_currentScenario.TestResult = testResultType.Failed
End Sub

 ''' <summary>
''' Scans the logs and checks if any failures have occured
''' </summary>
Public ReadOnly Property LogHasFailure() As Boolean
Get
If m_currentScenario IsNot Nothing AndAlso _
m_currentScenario.TestResult <> testResultType.Passed Then Return True

   m_summaryResults.TestResult = testResultType.Passed
For Each scenario As SummaryResultInnerTest In m_scenarios
If scenario.TestResult <> testResultType.Passed Then
Return True
End If
Next

   Return False
End Get
End Property

End Class

Disclaimer
This code offers no warranties.  If this code somehow causes health problems, or harms your computer in any way, please consult a physician and don't blame me (or Microsoft).