Pathological testing is where you test for conditions and inputs that you know full well are highly unlikely or improbable. Pathological testing is the bastard stepchild of a drunken one night stand between boundary testing and error testing. A simple example is the following:
You are prompted: Please enter your annual gross income: $__________
You enter: 10000000000000000000000000000000000000000000000000000000000000000000000000
While this response may fall into basic functionality testing if you’re Bill Gates or the Sultan of Brunei, in 99.999999% of cases it’s pathological. And it’s an oft-overlooked area of testing that can really serve your customers well. Yes, it’s hard to face the fact that your customers, the users, may be tempted to enter completely off-the-wall values to a prompt or may open a gazillion instances of the same form, but it happens.
A good way of doing basic pathological testing is to take any system limitation to an extreme. This differs from error testing because error testing simply tests expected error conditions, such as entering a string into an expected numeric input; it differs from boundary testing insofar as boundary testing generally consists of inputs at or closely exceeding telerance.
A good application must not only deal with simple errors, they must assume and trap conditions that could only be caused by a user who’s escaped from an insane asylum. It must also be able to recover from situations that couldn’t be trapped in real-time. Here are some examples of pathological testing:
- Your application pops up a window showing detail on a list of widgets when the widget is clicked on. If the window is non-modal, what happens when the widget is clicked on 1000 times?
- Right-clicking on a control fires off a process that takes a few seconds to complete. What happens if the user maniacly right-clicks that control dozens of times in that few seconds?
- You have a complicated database transaction that either has to totally complete and commit or is rolled-back. It usually takes several seconds. Start the transaction, then pull the plug on the computer.
- In the middle of a client-server or web-based app, pull the network cable.
While these are “pathological“ situations, it’s easy to see them actually happen, isn’t it? If your application can’t mitigate these situations in real-time, it has to have a recovery procedure to deal with them at next instance.
It’s easy to overlook issues like these and classify them as improbable and hope like hell they won’t happen, but a good application developer must have restriction, mitigation, and recovery routines in the application to deal with them if they occur.
Here at MS on the Fox Team, we’ve uncovered a lot of bugs pathologically. Some of these have been routine such as changed error messages at extreme inputs while others have been serious buffer overruns that could have compromised security. Other extreme values have caused crashes.
I’d like to hear back from y’all on pathological situations you’ve encountered and what the effect was or how you trapped them upfront.