Dangers of Test Automation Revisited

Here is a comment I received via e-mail in reference to my post on the Dangers of Test Automation.  I found it insightful.  The point is the oversights are easy and automation won't catch them.  Despite a few claims that this is just a deficiency in the test plan, assuming that a test plan is perfect is a dangerous assumption.  There are always going to be holes, just like there will always be bugs in the code we write.  Having manual exploration gives us insurance that we will cover these holes.  Once again, if all of your testing is automated, you are done finding bugs after your first run.

Here is the comment:

I was reading your post:

... and I was immediately reminded of a scene from Jurassic Park (the book, not the movie, though that was good too)

The park administrator is showing off the high-tech computer systems. He says "you can search for dinosaurs... I'll just search for all of them... and the big board will show you their current locations." Sure enough, there they all are.

Later it is revealed that, contrary to design, the dinosaurs are breeding. Someone asks why the extra dinosaurs didn't show up on the computer search.

Nedry answers (I'm paraphrasing) "The search allows you to enter an 'expected number' of dinosaurs, which helps it to run faster. It's a feature."

Premature optimization strikes again...