I am Brian Cost, the QA Manager for the Windows SDK team. You’ve heard from Program Managers on various topics and now it’s time for Test to post.
As Vista Beta 2 approaches, the Windows SDK team has deadlines and mini-milestones to meet before we will be able to release the SDK for public consumption. One of those milestones is something called Zero Bug Bounce (ZBB). Basically, it’s the point at which there are no active bugs against the product for the upcoming milestone, and any new bugs coming in are fixed within a one or two day window. It’s also a point at which there is furious developer activity fixing bugs and testers verifying the bugs are fixed. It’s also a point at which Program Managers should stay out of the way.
What else does Zero Bug Bounce mean? Well, as developers are fixing bugs at a higher rate than testers are finding them, eventually you get to zero active bugs. Then, for a very short time, testers bump it up a notch and start finding bugs and driving the active bug count up, which is the bounce. Eventually it evens out again and bugs are fixed immediately keeping the active count at zero
Here is a little background as to what happens with bugs during a typical milestone release, and by milestone I mean a span of time that’s just a few of months (e.g. November to February CTP release). In the beginning of a milestone, Program Managers are very busy writing up specifications (spec) and getting sign-off from various members of the team (dev & QA). Very few bugs are found during this phase because there really isn’t a lot of code to be run and tested.
Once specs are done, developers get busy writing all the code to implement the specs. Again, not a whole lot of bugs are being found, but more than when they were in the planning phase. As developers are coding up features and builds are coming out on a daily basis, a lot more testing occurs, and here is where most of the bugs are found. As developers are still busy developing new code, they don’t have time to fix all the bugs that come their way. This is not to say there are a huge number of bugs in the product, it’s just that the developer’s responsibility is to make getting to code complete a priority, and fixing bugs is #2.
Now as we get to the finishing stretch and features are coded up and complete, focus turns to making sure comprehensive testing of the entire product has been done, and any remaining bugs are found and fixed. Developers sole goal at this point is to fix, fix, fix and testers to verify, verify, verify in the drive to ZBB. Once that date arrives, and there are 0 bugs active, it’s a major event. It means that testing has covered what they need to and exercised new code, and developers have resolved remaining issues. You can see the light at the end of the tunnel for the final RTM build and the point at which we put the bits out so you can download them.
I hope that this has given you a little insight into what happens on the Windows SDK team and get ready for the Beta 2 release coming soon!