You have already used the bug tracking system (WIT) in TFS before. So, we integrated this bug tracking system with MTR so that you can file bugs with a single click from the runner itself. What’s the big deal about this you might ask?
Well, the big deal is that we not just create a bug for you from the runner itself, we also populate it with the test case that you have just been executing when you opened the bug. And that too in this neat little repro steps control with a tab of its own in the bug form. (Yes, we modified the MSF template to include some extra tabs in the default bug) So, all comments that the tester has written, notes she has taken, attachments that she has made are all uploaded automatically to the bug from the first step to the last failed step.
Now, how often have you hit a bug during testing that would just repro on your test machine but simply would not repro on the dev machine. And how often has it turned out that it was a configuration related issue – different service packs, different CPU usage, different memories, the list is endless. To ensure that the developer gets a complete picture of the environment where the bug was actually produced, we fill in system information about the test machine automatically into the newly created bug. Check out the “system info” tab within the bug form:
Now, even with all of these, what if the developer still cannot repro the bug? Perhaps it was a domain expert that filed the bug and not a technical tester. He/she may not have noticed subtleties that might have affected the system and caused the bug. Or the bug repro steps or description maybe woefully inadequate for the developer to produce a reasonable repro. In such cases, the video recording that the runner makes in the background during all test execution, will be considerably useful. Playing back the video will give the developer a very detailed and accurate picture of the exact steps leading up to the bug. This video is filed as an attachment to the bug
What if the test itself took 10 long minutes. Now, the failure is in the last step perhaps. The developer may want to look at only the failed step instead of the whole long video. Video markers are the way to go for that. Each time the tester marks a test step in the test case, a video marker is inserted with that timestamp in the video file. These markers are then populated against the test steps in the repro steps of a bug filed via the runner. When I click on these markers, I can play back from the point where the previous step was marked. This gives the developer finer control over what portions of the video file may be useful.
So, how do you like our actionable bug? Are there any other ways we could help developers get the maximum effective amount of information from the bugs?