Implementing Automated Software Testing


Elfriede Dustin sent me a review copy of Implementing Automated Software Testing (IAST), which she and her colleagues Thom Garrett and Bernie Gauf recently published. I am familiar with some of Elfriede’s work on test automation, so I was looking forward to an in-depth explanation of how she does test automation.

That is not what this book is.

Part way through IAST I realized that it describes the process the authors use to design and build test automation frameworks, not the details of their implementation. Once I wrapped my head around that, IAST started making sense to me. The authors lay out a sensible approach to designing a test automation framework and managing its implementation, one which bears a distinct resemblance to the heavier software design processes. While you can get that from any process book, the value in this book lies in the test-specific additions, such as detailed instructions for calculating the expected ROI from test automation, and reviews of test case data and logic.

One point the authors make repeatedly is that writing test automation is a software development project and should be treated like one. I am happy they do, for one mistake I repeatedly see product teams make is treating test automation as a second-class citizen that any yahoo can throw together. Software is software, regardless of who writes it.

I found other statements to be happy about as well: the importance of testing test infrastructure was discussed several times, as was the importance of explicitly deciding which tests to automate and which to not. I also found much to disagree with. Throughout the first half of the book, in fact, I found myself saying “No no no no no!” at least as much as I said “Yes yes yes yes yes!” Most egregious to me were the continual statements that manual testing “can hardly do” specific types of testing. Eventually I decided this meant that these types of testing were difficult to do manually, not impossible.

Mostly, though, I was confused what point the authors were meaning to make. Were they attempting to convince me that automated testing was important? Or sell me on a specific way to do automated testing? Or describe to me their process for managing the development of test automation? In the end I decided on the latter, however I am still not quite sure.

If your team wants to start up (or revive) a test automation project and does not know where to start, you might find this book helpful. Browse it in your local bookstore before you purchase it to see whether it fits for you. If you find that it does – or doesn’t – let me know why: michael dot j dot hunter at microsoft dot com.

Comments (5)

  1. Shrini says:

    The word "automated testing" is also "ambiguous". When people say "automated testing" they mostly mean "execution of some tests by another computer program instead of a human tester". Considering testing is much more than execution of specified tests  by another computer program – calling that is automated testing is a largely inappropriate.

    I prefer "computer aided test execution" instead. This is "too" precise and too long .. that is why most prefer shorter and ambiguous version automated version.

    To me the phrase "automated testing" is still a mystery

    http://shrinik.blogspot.com/2007/07/mystery-called-automated-testing.html

    Shrini

  2. Automator says:

    Shrini

    The book actually defines Automated Software Testing as "The application and implementation of software technology throughout the entire software testing lifecycle with the goal of improving STL efficiencies and effectiveness."

  3. Gary says:

    This sounds like a good resource for people that want to revamp or undertake automation with existing knowledge of how to do so.  And a good review of it too, thank you!

    Is there a book you would recommend for a very inexperienced programmer who has been in test for 6 years or so that wants to start delving into the complex world of test automation?  If it helps, my office works almost exclusively with web applications done in Flash.  I have had a really hard time finding resources to help with automation of Flash other than really rudimentary JS procedures.

  4. micahel says:

    Gary: I do not know of any books which cover these topics in the arenas you are looking for. I suggest visiting your local book store and browsing through the computer testing books; maybe one of them will fit the bill.

  5. For me automated testing is absolutely necessary for big software projects. TDD rultes.