Through The Glass Darkly

Testing is often divided into two types: black box testing, where you poke at the product from the outside and don't have a clue how it's put together or what makes it tick, and white box testing, where you inspect source code and execute the product under a debugger and have complete access to every last detail. Black box testing is certainly important, for this is the closest you can get to what the customer will experience. White box testing is very useful as well, because when you understand the assumptions your developers made you can identify how those assumptions might go awry. With black box testing you don't know the developer's model and thus are forced to form your own; it may have zero correspondence with the developer's, and so you may miss problems that would otherwise be obvious. On the other hand, with white box testing you can become too familiar with the developer's model and so lose the ability to see what's wrong with it - again missing problems that would otherwise be obvious.

I find that grey box testing provides a nice balance between these two extremes. Here you understand the developer's model well enough that you can push at it and see where it falls down, but at the same time you maintain sufficient distance that you don't lose your perspective. My favorite starting place is to ask my developers to explain their design. Generally I try to avoid doing this in their office, because a) most conference rooms have way more whiteboard space, b) it's a lot harder for them to just drone through the code when it's not right there, and (bonus) c) if they don't know their design well enough to explain it to me without constantly referring to their code I know it almost certainly isn't worth testing yet!

This works great for me, and I thought it would work for my team as well. So I asked our architect to take our next many design reviews to take us through Sparkle's design: what it is, why it is that, how it got there, and where it's going. It's been fabulous! We're all learning lots and testing better as a result.

We are doing this in special meetings, but talking with your developers over lunch or snack breaks works just as well. Eventually you may discover that you aren't just learning about your application but that you are also building a relationship with your developers. This is a Very Good Thing! Nurture that relationship, and you may find your developers asking you how they can better test their code. And then you're approaching nirvana!

*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great coding skills required.