scripted?

The debate over scripted vs. exploratory testing continues (at least among the exploratory testers). A mailing list I'm on has had some discussion on this recently, and have brought up a few really good points. Rather than respond on the list (froma "non-creative, far too technical, just don't get testing, script kiddy" email address), I thought I'd post my thoughts here for the rest of us).

One discussion started with this (more or less) question. "My boss / client / overlord wants me to do some testing. He has a set of scripted tests, and I want to do exploratory testing. There's only time to do one - which do I do? How do I get my boss / client / overlord to know that ET will be super-awesome?"

The answer to both questions, of course, is yes (or 42). Many years ago, on my first day of work at Microsoft, my boss (who was awful in so many ways) handed me a list of test cases and said, "Here, make sure you run these every day". As an experienced tester, I took that to mean - "if these things ever fail, we're in deep doo-doo, so make sure these work as well as can be expected, but while you're there, do some other testing as well".

I did it my way. After a few days, I got bored, and automated most of the test cases so I could think of other things to test. I randomized some data and varied some of the steps. Of the tests in the original script, I don't think any ever found a bug. But I found dozens (perhaps hundreds) of other bugs. But I was running tests from a script. Kind of?

The management / client / overlord pushback from the original question would be - "You're telling me you can either run my scripts that (to my knowledge) verify the features our customers expect to work are working, OR, you can play around with the app for a while?" If you put it that way, what do you expect them to say? If test scripts are run in tunnel vision, of course they will suck. If they're used as a guideline to make sure a few things are done during exploration, they can be quite effective. Will there be some situations (ahem - contexts) where pure ET will be superior to any guidance that a script or checklist can provide - you bet. I'm just saying (once again), that it's not a one or the other choice, and that good testers know how to make this balance (as well as communicate the choices to their management).