Testing != Bug finding

It’s probably been close to five years since I was in a room filled with twenty or so test leads and blurted out “it’s not the job of test to find bugs”. On that particular day, you could have heard a pin drop, but over time, that team – as well as other teams I’ve worked with are realizing this more and more. I’ve talked about the role of test before, so I’ll try not to rehash the same thing. I just get irked when I keep seeing web and print articles touting bug finding techniques as the essence of testing.

I have a bruise on my forehead from banging it against the wall after reading James Bach’s latest blog post. The summary is that he doesn’t want developers to run unit tests because then he won’t be able to find as many bugs (I guess this keeps him in business?). Defect prevention is where the real money is. Short of prevention, (almost) everyone knows that it is orders of magnitude less expensive to find defects as close to their introduction as possible. You may have heard the folk tale of the people drowning in a river (several variations exist):

Several villagers were walking by the river. One of the villagers noticed someone in the river yelling for help.

One of the villagers swam to save him. Then, they noticed another man in the river, and they pulled him out too. Soon, more people were seen drowning in the river, and the villager were pulling them out as fast as they could. They began to organize their rescue efforts and tried to find the most efficient method of saving everybody floating by. In the midst of this, two of the villagers started to run away along the shore of the river.

"Where are you going?" yelled one of the villagers. "We need you here to help us save these people!"

"We are going upstream to stop whoever is throwing them in!"

Anyone can hang out on the bank of the river and save hundreds of lives. You may even be good at coming up with better techniques for saving drowning people. A professional tester, however, always knows where to focus their efforts.

On a related note, the September issue of Better Software will print my short article titled “I’m tired of finding bugs”.

Comments (5)

  1. james bach says:

    Hi alanpa,

    I’m sorry, but you seem to have misunderstood my blog post. I did not say that I don’t want developers to run unit tests. Actually, I’m happy if they run unit tests.

    The thrust of my message was that I don’t want developers to delay my work by thinking that I don’t WANT the product until they have already tested it. My work is to discover important information about the product. Mainly, I find information about bugs in the product. I do other things, but bug finding is a big part of my work.

    As I said in my blog, if the developers want to test, that’s fine. The thing is, developers are generally bad testers, whereas I love testing, and I’ve made it my career to study how to do it better than almost anyone else. So, I advise developers to let me at it.

    — James

  2. Alan says:

    OK – fair enough explanation. Our disagreement is in two areas. One is that I think developers can and should be good testers. I don’t want to spend time finding simple functionality bugs, logic errors, off by one errors, etc. Any bug that I can find in less than a minute should have been found by the developer.
    This, of course, doesn’t mean that I can’t start using and analyzing the software early (or concurrently with the developer), but when I get something that is “ready to test”, the only bugs I want to find are issues in complex customer scenarios, bugs in component interaction, security issues, load issues, and other sorts of defects that aren’t easily caught by unit testing.
    I’m also someone who loves to test, but I prefer to do whatever I can to prevent developers from creating bugs rather than finding them myself.
    Fundamentally, I guess we agree – the sooner we get our hands on code, the sooner we can provide meaningful information to development and other stakeholders – I just have a slightly different take.
    Thanks for stopping by to comment – I appreciate the feedback.

  3. I. M. Testy says:

    For the past few years I have been teaching new testers at Microsoft. Before beginning the Microsoft…

Skip to main content