Missing States

Every tester forms a model of each application they test. Sometimes this model is explicit, such as when the tester is doing model-based testing. Other times it is unconscious, such as when a tester does not know why they do what they do to find problems.

Similarly, every application functions according to a model. Sometimes this model is explicit, such as when the application is built on a state machine. Other times it is unconscious, such as when it is built by a developer who does not know why they do what they do.

Every customer forms a model of each application they use. Sometimes this model is explicit, such as when the customer explains to someone else their view of why the application works the way it does. Other times it is unconscious, such as when the customer accretes a set of magic steps which get the application to do what they want it to do.

Sometimes these models match up. More often they are wildly different.

The customer's model matching the developer's model tends to be a Good Thing. If the customer's understanding of how and why the application does what it does matches the way the developer designed the application, the customer will know how to make the application do what they want it to do, and they will rarely be surprised by it. The extent to which the customer's model diverges from the developer's model tends to indicate the extent to which the customer will be annoyed and frustrated as they attempt to use the application.

The tester's model matching the developer's model is a Good Thing insofar as it indicates each has a similar understanding of how the application is supposed to function. It can also be a Undesirable Thing insofar as the tester gets trapped by model blindness.

Model blindness occurs when a tester forgets that a model can only contain those states which are expected to happen. Some of the states will be standard behavior; others will be for error conditions and other rare occurrences. All of them have been considered, else they wouldn't be part of the model. While verifying that these expected states and the transitions between them is important, a tester who limits themselves to only these states is, as Shrini reminded me, missing a target rich environment.

This is one reason that cross-feature and integration testing tends to find copious amounts of gnarly issues - while most developers are able to maintain a model of their feature in their head, doing the same for every other feature as well becomes progressively more difficult the more other features there are. So they miss things.

One way to avoid model blindness is to look at only the states, ignoring the transitions, and search for ways to get from any one state to any other state. Another is to look for additional paths for reaching those states. Your favorite get-out-of-the-box-I-am-in technique likely works well here too.

Model blindness can occur even when your model is different from that of your developer. If you are working from a model you have excluded certain items. Unexcluding one or more of these will probably find issues.

Assuming you're working from a model.

Which you always are.

*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.