Preemptive Testing

Chris posted a great comment a while back that I am going to mine for a series of posts.  He starts out by saying "I'm struggling with the contribution that testers make early in a project right now." The answer is simple:  a lot, if you let them. 

"Everyone knows" that testers can't do anything until they have something to test. (C.f. what they evidently think over at Bungie over at Bungie, which is otherwise a great description of testing.)  Therefore it goes without saying that testers are pretty much useless until Dev has written enough bugs (erm, code, I should have said) that there is something worth testing. This puts Test perpetually behind the curve with no chance of ever catching up. While Dev has moved on to the next milestone Test is still banging on the previous one. Any bugs they find are often ignored until Dev can bear to pull themselves off their fun new work back onto all that boring "completed" code they wrote last milestone. The only reason the product ever ships is that eventually the Devs hit code complete and (theoretically, at least) stop writing features and focus completely on fixing bugs.

Slightly more enlightened teams let the testers see the specs once they are complete. Test is still behind the curve, but at least they can start writing their test plans during that early period when they don't yet have anything to test.

Some teams actually deign to involve Test in the specification process. We testers are adaptable creatures just raring to test anything we can get our hands on, and we can test specs just as easily and as well as we test code. It's a rarely-argued fact that finding bugs earlier saves beaucoup time and effort, so set us loose on the specs and let us find them at the earliest possible moment (i.e., in the spec). There still will be bugs in the code that results from those specs, though, so Test is still left behind the curve trying to catch up to Dev.

The only way to get ahead of the curve is to figure out how to start writing tests before there is anything to run those tests against.  This is what executable specs and modeling languages like AsmL are all about.  Not all problems are amenable to such tools, but even so there's usually something you can do here.  If you're testing a COM object model, you can start writing tests as soon as that object model is stubbed out.  If you're testing UI, you can't actually run anything until that UI is defined, but you can write test cases for how that UI should behave.  (Assuming, of course, the UI is defined in the spec.  If it's not, you have a bunch of spec bugs to write!)

If you're an Agile-ish team, your testers would *love* to help your customers write acceptance tests.  If you are Test Driven Design-ing, your testers would be more than happy to help you define those tests as well.

Assuming your team is willing to include Test throughout the product cycle, then, the "contribution that testers make early in a project" is:

• Test the stuffing out of the spec.
• Draft tests right away, and build/run them immediately when functionality finally shows up.
• Provide early guidance as to the testability of the feature and code.  (Oops, I haven't talked about that yet.  That's a post or two all by itself...coming soon!)

If you aren't willing to treat Test as equal but rather prefer to lob code at them periodically and without warning (why make it easy on them, after all), you don't have a Test team at all.  (This is my biased opinion, remember; please read my disclaimer.)  Not for long, at any rate, as they are almost certainly looking for new jobs!

*** Comments, questions, feedback?   Want a fun job on a great team?  Send two coding samples and an explanation of why you chose them, and of course your resume, to me at michhu at microsoft dot com.  I need a tester and my team needs a data binding developer, program managers, and a product manager.  Great coding skills required for all positions.