Test Results in Build

Anand Kamat

November 18th, 20150 0

Back in June 2015, we introduced the ‘Runs’ under Test hub as a part of the Automated Testing Workflow with the new TFS Build system. Runs hub was designed to be a repository for all test results generated in the system, helping you browse test results from various workflows such as Build, Release Management, Manual testing, and results published using REST APIs.

To further enrich the test result analysis experience, we have updated the Build summary page to show test results. This experience, which is ‘in-context’ of where the tests are run, has rich insights that will help you in debugging failed tests. If you have builds that run tests, then this experience is available for your recently queued build. Open the build summary page and click on the Tests tab to get started. If you don’t have tests in your builds, get started by adding test tasks to the build definition.

Features and insights that are lighting up with this experience

Aggregate status of tests and delta with previous build:

Aggregate summary for total tests, failed tests, pass percentage and test duration, giving you a consolidated status across all test runs.
Break up of test failures into ‘New’ and ‘Existing’ failures. New failures are tests that were passing in previous build but are failing in the current build. These will help you identify regressions introduced in the current build.

Failed tests with relevant insights:

New test failures are highlighted as New, bringing such failures to your attention and helping you prioritize which tests to debug first.
For existing failures, you can find out how long the test has been failing, and in which build was the failure introduced. By navigating to the failing build, you can look at the check-ins/commits that went into that build and may have caused the test to fail.
You can group test results by pivots such as Priority, Container and Test run. We plan to add support for more group by pivots soon.

Detailed information for a failing test

You can find the error message and stack trace in the details pane to investigate a failed test.
Recent outcome trend will help you quickly identify if a test fails intermittently.

Create and view bugs

You can file a bug for a failed test. The bug work item will be auto-populated with information like error and stack of the failing test.
You can view the bugs associated with the test in the the details pane.

What’s coming soon?

This experience works great with the new build system (Build vNext). Support XAML Builds is coming soon.
Capability to view all test results, including passed tests, and option to filter test results by outcome.
And many more interesting features…

This feature is now available in ‘preview’ on Visual Studio Team Services (formerly Visual Studio Online). We plan to ship this feature for on-premises TFS users with TFS 2015 Update 2. We are keen to hear your feedback. Please share your feedback in the comments below, or email your feedback to devops_tools@microsoft.com.

Happy debugging,
Manoj Bableshwar – Visual Studio Testing Tools

(Chass editors note: we also look at this in this Channel9 video: https://channel9.msdn.com/Series/Test-Tools-in-Visual-Studio/Unit-Testing-from-a-CI-Build-with-Visual-Studio-Team-Services)