Triaging our "tools" bug list

Last week we took a look at bugs we have opened in the "Tools" branch of our bug database. For OneNote, "tools" is a catchall for a wide range of differing areas: automation bugs, performance work needed, bugs with out test tools (which is probably the most straightforward use of this bucket), automation backbone bugs and so on. Generally speaking, these are not bugs with OneNote, but with the testing system around it.  We wanted to go through these to ensure they didn't start to accumulate and start to become a huge work item some time in the future.

Our triage process was fairly informal. We wanted to determine if the bugs needed to be fixed, needed more information before we could decide or if we could not fix the report. Not fixing "bugs" always sounds bad, but for this triage process, if a workaround was available, we might have decided to live with the workaround rather than investing in a proposed fix.

And as it turned out, we had only one bug we said "No" to. It was a request to be able to rename a section in an automated test with only one command - something like this:

Section.Rename("My New Section Name");

There was a workaround:

Section.Rename();

Section.ApplyNewName("My New Section Name");

At this point, we closed the bug - it's just one extra line of code in a test script to workaround the reported problem. Plus (and this has nothing to do with the technical details), this more closely mimics the user steps of choosing to rename the section, then hitting ENTER when done. In any event, it was an easy decision to leave the bug unfixed.

A bug we decided to fix was copying some VS project files automatically to the Projects folder on your local hard drive when you enroll in the test project. This way, you can click File | New | OneNote Test script and avoid all the manual copying and registration which you would otherwise need to perform. This saves a good bit of time for every tester on the team, so we decided the payoff was worth the effort.

We will get a complete list of the bugs we need to fix after everyone fills in the "more information" we requested.  Then we will estimate how much time it should take to fix them. Once we have that estimate, we can look at each testers' schedule to see how much time she has to devote to these fixes, and assign them to avoid overloading any one tester.  This proactive approach will help prevent any scheduling conflicts (aka "fire drills") in the future.

Questions, concerns, comments and criticism always welcome,

John