Diagnostic Data Adapters: Changing how Developers and Testers work together (Part 2 of 2 - The Diagnostic Trace Collector)

This post finishes up the two part topic on Diagnostic Data Adapters in VS 2010, and how they help developers and testers to work together. In the last post I talked about the “Test Impact Collector”, and today I will talk about the “Diagnostic Trace Collector”.

In an earlier post, I had talked about the “No Repro” problem that plagues the Industry today. This is that frustrating, almost reflexive, resolution to bug reports by developers – or perhaps the only option left when the test team didn’t provide enough information in the bug report to make heads or tails of what they were trying to do with the product. The VSTS 2010 product provides a variety of mechanisms and dials to combat this problem. The Diagnostic Trace Collector is another one of those, and can turn any test case into a data record complete enough to support time travel debugging! Any tester can run a manual test with the Diagnostic Trace Collector enabled and file a bug. From that bug report the development team will have enough data about the state of the system under test to walk backwards and forwards from the point of test failure in a debugger located on their own machine.

This is a very powerful feature! Visualize this picture. The Developer simply clicks on a log file attached to a bug that’s assigned to him, and this launches Visual Studio into a debug like state that allows him to inspect call stacks, events, function parameters and the like. The magical part of this of course is that you get the experience of Visual Studio debugging an application that is no longer running on the developers work station or the testers machine, yet it literally is the state of the application at the time the defect was filed, and the developer can now trace backwards in time to see how the defect occurred!

Below is a screen shot that shows the configuration options for the Diagnostic Trace Collector.

This  Collector is a natural to enable when the development team asks for more information regarding a specific bug report. While the default settings collect events raised by the applications under test, these applications can also be instrumented to facilitate tracing method level data to provide a more detailed picture of the state of the system during the test. The result file will be automatically included with any bug filed from the test runner when the test settings enable the collector, and will appear in the result file for any test run from Visual Studio with similar settings. Most of our diagnostic data adapters are available to tests executed through Microsoft Test and Lab Manager as well as the Visual Studio integrated development environment.

Double clicking the log file in a test result or a bug report will open the Trace Debugging Log viewer, shown below with a display of active threads running user code. This viewer is the access point to historical debugging based on the data collected during a manual test, for instance.

So, instead of a No Repro, any tester can now create a debuggable record of the test they performed with no special technical requirements necessary to collect this data!

Now that we’ve seen what a couple of these diagnostic data adapters can do we’re ready to go about authoring one of our own. We’ll tackle that challenge next time. The great part about our whole Agent-Controller and Diagnostic Data Adapter system is that teams can use the diagnostic data adapters we include or write their own to get all the utility they need out of their agents without installing the Visual Studio 2010 product on the agent or controller machines. It is as simple as dropping the new adapter assemblies into the correct folder and updating the test settings to make use of the new functionality – or simply using the out of the box adapters as they are. We’ll be going through these details and more next time, please do ask any more questions you may have as I want you to be as excited about this topic when you get your hands on the bits …