Using the TFS Object Model to populate Test Details
Comprehensive documentation on the various TFS Object Models is available on MSDN here. While the documentation is complete, sometimes it helps to see a sample to help you understand how to use a tool. In this post, I will show you how to do that. Say you are using running manual tests and you want information about your test to be saved as results for the test case in the Microsoft Test Manager tool under the Details section circled below.
So how do you write a little app that uses the TFS Object Model to help you enter these results? That is what I am going to discuss today.
The application is pretty simple. Most of the major functionality exists in a class I’ve called TestCaseDetails. It connects to a TFS Server and Project that you specify with your credentials.
Once you do that, you can press the “Connect…” button to verify your information.
// Try and connect to the Project on the server
tcd = new TestCaseDetails(server.Text, project.Text);
bool bOK = tcd.InitConnection();
If all is well, you will be rewarded with the following dialog.
If there were problems connecting, then an error dialog will be displayed.
Once you have a successful connection, click on the “Test Case” tab at the top of the dialog and wait for the Test Plan, Test Suite and Test Case dropdowns to populate. Test plans are populated first and the first test plan in the list is selected, followed by the first test suite and the first test case. Each of these is populated by a method call to the previously created TestCaseDetails object tcd that returns a simple list of strings. Each of these is assigned to the DataSource of the proper control. For example:
testPlansList.DataSource = tcd.GetTestPlans();
If you have many items this will take some time to populate. Once the dropdowns have populated, you should see something like this:
Here you can change the Test Plan, Test Suite and Test Case you would like to modify. Once you have made the proper selections, you can click on the “Add New Results…” button. This button assigns the dropdown selections you made as well as the TestCaseDetails object to member variables in the class GenerateResults and displays the following dialog:
Here you can add in the information you require for your manual test run. The fail error message will only be placed on the step that fails; if the test steps passes, then you need an error message displayed. Pressing the “Save” button commits all of the information to TFS.
Now let’s take a look at some code so you can see what is going on under the hood when you press that “Save…” button. A call to the CreateTestRun method is made on the TestCaseDetails object. Let’s look at the code for that method:
public void CreateTestRun(string testPlan, string testSuite, string testCaseID,
string testCaseNotes, DateTime startTime, DateTime endTime,
string computerName, TestOutcome testOutcome, string comments,
string msgFail)
{
// Retreive interfaces to the test plan and suite
ITestPlan plan = getTestPlan(testPlan);
ITestSuiteBase suite = getTestSuite(testSuite);
// Now retreive the collection of test points
ITestPointCollection tpc = plan.QueryTestPoints(string.Format(
"SELECT * FROM TestPoint WHERE TestCaseId = {0}", testCaseID));
// Create a new test run
ITestRun run = plan.CreateTestRun(false);
// For each test point in the colletion, add the person who ran the test
foreach (ITestPoint p in tpc)
{
run.AddTestPoint(p, tfs.AuthorizedIdentity);
}
// Save the test run
run.Save();
// Generate the test results
GenerateResults(run, testCaseNotes, computerName, testOutcome, comments,
startTime, endTime, msgFail);
}
Here we create the new test run and add in each test point for the given test case ID. Once this is completed, the code moves to the GenerateResults method shown below.
private void GenerateResults(ITestRun testRun, string testCaseNotes,
string computerName, TestOutcome testOutcome, string comments,
DateTime startTime, DateTime endTime, string msgFail)
{
List<ITestCaseResult> testCaseResultList = new List<ITestCaseResult>(1);
// Find the number of test case results
ITestCaseResultCollection testCaseResults = testRun.QueryResults();
foreach (ITestCaseResult testCaseResult in testCaseResults)
{
System.Diagnostics.Debug.Assert(testCaseResult.TestCaseId != 0);
// Execute the test case
ExecuteTestCase(testCaseResult, computerName, testOutcome, comments,
startTime, endTime, msgFail);
// These are the test result notes and who ran the test
testCaseResult.Comment = testCaseNotes;
testCaseResult.RunBy = tfs.AuthorizedIdentity;
testCaseResultList.Add(testCaseResult);
project.TestResults.Save(testCaseResultList.ToArray(), false);
}
// Even if there was nothing to run, save the results
// Saving a zeroed list is required if nothing exists
if (testCaseResultList.Count != 0)
{
project.TestResults.Save(testCaseResultList.ToArray(), false);
}
}
In this method, we create a new List of ITestCaseResults with at least one item in it. Then we query the test run for the result collection so we can execute each test case in the run. If there are multiple test case results, we add them all to this master list we previously created and save the entire list to the test results. If a test case result does not exist, you need to add one to the test results. If you do not, nothing will be shown in the Microsoft Test Manager for this run.
Executing each test case is done in the next method called ExecuteTestCase. The code for it is shown below.
private void ExecuteTestCase(ITestCaseResult testCaseResult, string computerName,
TestOutcome testOutcome, string comments, DateTime startTime, DateTime endTime,
string msgFail)
{
ITestCase testCase = null;
// Get the test case
testCase = testCaseResult.GetTestCase();
// Get the number of iterations
int numIterations = testCase.DefaultTable.Rows.Count;
// If there are not any iterations, then execute the test case at least once
if (numIterations == 0)
numIterations = 1;
// Note the start time, computer this was run on and the outcome
testCaseResult.DateStarted = startTime;
testCaseResult.ComputerName = computerName;
testCaseResult.Outcome = testOutcome;
// Loop through for as many iterations as necessary.
for (int j = 1; j <= numIterations; j++)
{
// Create the assignment
ITestIterationResult testIterationResult = testCaseResult.CreateIteration(j);
testIterationResult.DateStarted = startTime;
// Simulate test results for each step
foreach (ITestAction testAction in testCase.Actions)
{
// Execute each test action
ITestActionResult testActionResult =
ExecuteTestAction(testIterationResult, testCase,
testAction, testIterationResult.Actions, startTime, endTime,
testOutcome, msgFail);
// Did the test pass?
if (testActionResult.Outcome != TestOutcome.Passed)
{
testCaseResult.Outcome = testActionResult.Outcome;
// Cause the test case to stop executing.
j = numIterations;
break;
}
}
// These are Test Result Summary Notes
testIterationResult.Comment = comments;
// Generate the time span for the duration
testIterationResult.DateCompleted = endTime;
testIterationResult.Duration = endTime - startTime;
//Save the assignment result
testCaseResult.Iterations.Add(testIterationResult);
}
// Fill in the date completed and the duration.
testCaseResult.DateCompleted = endTime;
testCaseResult.State = TestResultState.Completed;
testCaseResult.Duration = endTime - startTime;
}
By executing the test case, we start to fill in some of the additional information that will be reflected in the Microsoft Test Manager user interface. We start by getting the test case for the requested result and then return the number of iterations. If there are no iterations, we need to assign one as each test case must be executed at least once for the test case to pass. We assign the start time, computer name this test case was run on and the outcome of the test case up front. Then we assign the details of all the iterations required for the test case. For each iteration (including the one we are going to create), we again note the start time and then execute each test action by calling ExecuteTestAction. If the test action fails, we are going to fail the test case, so there is some logic here to short circuit executing the rest of the test actions. This can be overridden if you desire. For each iteration, we also we save the test result summary notes, the completion time, duration and add the result to the list of iteration results. Once all the iteration information is completed, we assign the completed time, state and duration for the test case result.
There are a couple of assumptions I am making to simplify this tool. First, all iterations in this tool will have the same information displayed on them. Second, the iteration results for outcome and duration will be the same as the entire test run result. Obviously, if you had multiple iterations, each of them would have a unique timing associated with them and would then result in a test run that encompassed the start and end times of all those iterations. You are free to change this if needed.
ExecuteTestAction is divided into three parts; executing group steps, shared steps and individual steps. In this sample, I am again ignoring shared steps. The code for ExecuteTestAction is below.
private ITestActionResult ExecuteTestAction(ITestIterationResult testIterationResult,
ITestCase testCase, ITestAction testAction, TestActionResultCollection collection,
DateTime startTime, DateTime endTime, TestOutcome outcome, string errMsg)
{
// Figure out how to deal with the test action.
ITestActionGroup testActionGroup = testAction as ITestActionGroup;
// If this is a group step execute it
if (testActionGroup != null)
return ExecuteGroupStep(testIterationResult, testCase, testActionGroup,
collection, startTime, endTime, outcome, errMsg);
ISharedStepReference sharedStepReference = testAction as ISharedStepReference;
// If this is a shared step, ignore it
if (sharedStepReference != null)
return null;
// Otherwise, return the single step execution test result
return ExecuteTestStep(testIterationResult, testCase, testAction as ITestStep,
collection, startTime, endTime, outcome, errMsg);
}
ExecuteGroupStep iterates over all the test actions in that group and calls ExecuteTestAction to get to the single step case. This allows for discovery of nested groups if needed.
ExecuteTestStep creates the result for the test step, records the start time, end time, duration and outcome into the result before saving the result into the collection. Additionally, if a step fails, you can set the error text associated with this failed step here. The code for ExecuteTestStep is shown below.
private ITestStepResult ExecuteTestStep(ITestIterationResult testIterationResult,
ITestCase testCase, ITestStep testStep, TestActionResultCollection collection,
DateTime startTime, DateTime endTime, TestOutcome outcome, string errMsg)
{
// Create the test step result
ITestStepResult testStepResult =
testIterationResult.CreateStepResult(testStep.Id);
// Record the time started.
testStepResult.DateStarted = startTime;
// Get the result's outcome.
testStepResult.Outcome = outcome;
// If the test fails, provide an error message.
if (testStepResult.Outcome != TestOutcome.Passed)
{
// This is shown under Test Step Details as the error message if a step fails
testStepResult.ErrorMessage = errMsg;
}
// Fill in the date completed and compute the duration.
testStepResult.DateCompleted = endTime;
testStepResult.Duration = endTime - startTime;
// Add the test step result to the collection
collection.Add(testStepResult);
// Return the test outcome.
return testStepResult;
}
That completes the sample. I hope it was valuable to you! You can download the source to this solution below.