That I am able to spend time blogging about Microsoft Identity Lifecycle Manager “2” has nothing to do with some scary marketing person suddenly declaring that we are allowed to talk about the product now, when we weren’t able to before. We’ve been public about the product in general, and most of its intended features since we annouced it at the RSA Conference in San Francisco in 2007.
Rather, I have had blogging time recently because we now have no less than 138 test applications that have to be run before we can submit any code changes, and several of those test applications incorporate many individual tests–26 in the case of one of mine that I had occasion to spend a lot of time on over this past weekend. As a result they take a while to run, during which time, I can, say, write a blog post.
Actually, our rule is that we have to run the tests on an essentially clean machine. Such a machine does not have our source control client installed on it. We copy over the binaries from our development machines, (re-)install the product, copy over the test sources, and run the tests. When we are at that stage of testing, obviously, we can be more productive during the test runs, working on our development machines while the tests proceed on the clean machines. However, especially in the case of more complex changes, it is generally a good idea to run the tests, or at least the most pertinent subset of them, through on our development machines first, where we have more facilities for debugging if any of the tests fail. That’s what’s happening as I’m typing this entry–a large subset of the tests are ticking over on my development machine.
Naturally, now that we are approaching the completion of our release candidate, nothing is more important than avoiding regressions. So the development team has to take particular care not to introduce any new defects, and to endeavor to catch any issues before the next daily build goes to the test team.
Most of the developers are currently in the state of what we call “Zero Bug Bounce,” or ZBB, which is reached when your list of bugs for the current release drops to zero. Inevitably, one is going to “bounce” up again as the test pass identifies more issues, but the idea is that once one has hit ZBB, the duration of each bounce will be briefer, and that bounces will soon be stopping. I hit ZBB myself for this milestone on Monday morning, then identified 4 new issues myself during the course of yesterday, plus one that got added by a test team member this morning. So my bug count is currently sitting at 5. I have made all the fixes, and I am waiting on the outcome of the aforementioned tests before distributing the changes for code review, and running the full suite of tests on my clean machine.