Pathological testing for appdevs (by John Koziol)

Pathological testing is where you test for conditions and inputs that you know full well are highly unlikely or improbable. Pathological testing is the bastard stepchild of a drunken one night stand between boundary testing and error testing. A simple example is the following:

You are prompted: Please enter your annual gross income: $__________

You enter:              10000000000000000000000000000000000000000000000000000000000000000000000000

While this response may fall into basic functionality testing if you're Bill Gates or the Sultan of Brunei, in 99.999999% of cases it's pathological. And it's an oft-overlooked area of testing that can really serve your customers well.  Yes, it's hard to face the fact that your customers, the users, may be tempted to enter completely off-the-wall values to a prompt or may open a gazillion instances of the same form, but it happens.

A good way of doing basic pathological testing is to take any system limitation to an extreme.  This differs from error testing because error testing simply tests expected error conditions, such as entering a string into an expected numeric input; it differs from boundary testing insofar as boundary testing generally consists of inputs at or closely exceeding telerance.

A good application must not only deal with simple errors, they must assume and trap conditions that could only be caused by a user who's escaped from an insane asylum.  It must also be able to recover from situations that couldn't be trapped in real-time. Here are some examples of pathological testing:

  • Your application pops up a window showing detail on a list of widgets when the widget is clicked on. If the window is non-modal, what happens when the widget is clicked on 1000 times?

  • Right-clicking on a control fires off a process that takes a few seconds to complete.  What happens if the user maniacly right-clicks that control dozens of times in that few seconds?

  • You have a complicated database transaction that either has to totally complete and commit or is rolled-back.  It usually takes several seconds.  Start the transaction, then pull the plug on the computer.

  • In the middle of a client-server or web-based app, pull the network cable.

While these are “pathological“ situations, it's easy to see them actually happen, isn't it? If your application can't mitigate these situations in real-time, it has to have a recovery procedure to deal with them at next instance.

It's easy to overlook issues like these and classify them as improbable and hope like hell they won't happen, but a good application developer must have restriction, mitigation, and recovery routines in the application to deal with them if they occur.

Here at MS on the Fox Team, we've uncovered a lot of bugs pathologically. Some of these have been routine such as changed error messages at extreme inputs while others have been serious buffer overruns that could have compromised security. Other extreme values have caused crashes.

I'd like to hear back from y'all on pathological situations you've encountered and what the effect was or how you trapped them upfront.

Comments (9)

  1. Girish says:

    Hi, My wife, in her previous incarnation, used to be a QA tester. And she would do this kind of thing all the time. Of course, It was not good that she brought that over even after she moved to a developer position. In fact she would even test random web sites.

    Once on one of the trading web sites, she tried typing in 255 letter user name and some similar sized password. Of course the UI could not show the user name.

    Also, in my windows 3.1 control development days, I used to add some backdoor key strokes to exersize some of the edge conditions. Such as making the window sizes enormous or tiny or moving them off the window so on.

    It was a a lot of fun. Nowadays, I find managers almost discouraging such testing.

  2. raj says:

    One of the things we look out for is trimming the input data (after we run it through a regex validator) becuase a user might enter




    in the password field.

  3. matthew says:

    i did this the other day. I went to rentacoder, and got pissed off with the coder who was whining about extending the deadline. I put in 10000000000 days (to be infinite, to shut them up) – crash. I put in a smaller value – saves ok, but job cannot be viewed due to overflow errors (which were neatly displayed to the user, instead of having a nice custom error page)


    PS I don’t think they bothered to fix the bug – a lot of programmers are arrogant, and think that this is user error, and not in fact a software error.

  4. Joku says:

    Well one often done thing (when bored or pissed) is pressing win-e continously to bring (too many) explorer’s. Or similar things, like 20-30+ IE’s when I’m too lazy to shut them down. What happens is every application starts to show problems as dialogs may stop working and whatever. Most annoying thing about is that I still have a ton of memory free but the OS can’t allow me to open more IE’s, hopefully fixed in LH.

  5. John Koziol says:

    Girish, you’re finding managers who discourage this sort of testing?

    ‘Bad manager!’, I say, whacking on the butt with a rolled-up newspaper. ‘Bad, bad manager!’

    It sounds to me like you and your wife are doing precisely the right sort of testing. Sometimes, I think, there is a tendency in project management to "ship at all costs" once milestones have been breached. It’s not good for the product nor the customer.

  6. John Koziol says:

    Matthew, your experience is the perfect example of what I’m talking about. Once you’ve exceeded a boundary, the error trapping should be consistent whether you’ve exceeded by 1 or 1 billion.

    As to arrogant programmers…hmmm. Gosh, I don’t think I’ve ever seen that. Every developer I’ve ever encountered was humble and accepted responsibility for pathological bugs. In fact, they often volunteered to shine my shoes or serve me lunch. <bseg>

  7. Nicole Calinoiu says:

    My personal favourites are the various feline input tests (real cat optional <g>). Some examples:

    1. Sitting on keyboard.

    2. Launching from keyboard.

    3. Kneading the keyboard.

    4. Taking a stroll across the keyboard.

    5. Batting at a dangling mouse.

    Touch-screen interfaces also add some interesting possibilities…

  8. John Koziol says:

    I have 2 cats. Once, years ago, I was moving cross-town and didn’t have a cat carrier handy so I just put the cat in the car and away we went. Well, this cat had an inner ear imbalance and was slightly crazy. And it freaked out in the car and jumped on my leg where it proceeded to use my leg as anchor by sinking it’s claws as deep as possible. I bled like a stuck pig but there was nothing I could do until I reached the new house. At that point, my wife and I gingerly disengaged the cat but I must have lost 2 pints of blood and I had scars that lasted for months.

Skip to main content