The purpose of software testing
For the past few years I have been teaching new testers at Microsoft. Before beginning the Microsoft ‘assimilation’ process I always ask each new group to define testing in one sentence and list the primary objectives of software testing. Invariably, the majority of folks new to software testing believe the purpose of software testing is to ensure quality and the primary objective is to find bugs. Unfortunately, this is an antiquated view of testing, and perpetuating these limited perspectives of testing simply retards the role of testing from maturing into a professional discipline in the field of computer science.
Testing doesn’t ensure quality
The purpose of software testing is to assess or evaluate the capabilities or attributes of a software program’s ability to adequately meet the applicable standards and customer needs. So, when I hear a tester say the purpose of testing is to ensure quality I immediately ask them, “how can testing ensure quality?” Think about this rhyme for a moment:
I don’t get to drive the train,
I don’t get to ring the bell,
But, if the train jumps the tracks
See who catches hell!
I don’t get to discuss the needs of the customer, I don’t get to translate those needs into requirements, I don’t get to design or code the solution, and when I find a bug in the software I don’t even get to fix it. I can’t even demand that every bug I find gets fixed. In fact, I know that a number of reported bugs (including ones I think should get fixed) simply won’t get fixed. So, how in the hell can I ensure quality?
What I can do as a professional tester is to design the appropriate tests to evaluate the attributes and capabilities of a software program’s ability to handle valid and invalid inputs under various conditions. This approach not only has the potential to demonstrate a program’s ability to meet customer needs and expectations, but also includes identification of flaws or deficiencies that could negatively impact the customer (or long term maintenance of the product).
The purpose of testing is not to find bugs
The primary objectives of testing include:
- Qualifying a software program’s quality by measuring it’s attributes and capabilities against expectations and applicable standards
- And (probably the most important role of testing is simply to) provide information
Although I vehemently disagree the primary objective of software testing is to find bugs, I am not surprised when new testers state bug-finding as a primary purpose of testing because it is so heavily emphasized in many books and proselytized by so-called ‘experts’ in the field. Even Cem Kaner states in his “best selling” book on software testing, “A test that reveals a problem is a success. A test that did not reveal a problem was a waste of time.” Bugs are simply a by-product of testing.
So riddle me this, if testers are spending all day beating on the product in attempts to find bugs, who in the hell is verifying the product does what it is supposed to do? I guess performance testing is a waste of time if the performance measurements never fall below acceptable levels. But wait, how would we know if we didn’t “waste the time” to measure performance to begin with? I guess we would have to waste time to actually measure the capabilities against the expectations or applicable standards, or say…”yep, in my ‘expert’ (biased and subjective opinion) it’s good enough!
My teammate Alan also recently wrote an excellent post entitled Testing != bugs, where he talks about pushing quality upstream through defect prevention. This is a common theme we discuss in our training for new software testers. We advocate the expansion of the testing role beyond bug-finding. We encourage testers to explore opportunities, seek out solutions to difficult challenges, and unlock their potential to become a greater asset to the organization throughout the entire software process. Fortunately, other companies besides Microsoft are beginning to understand that a highly trained and competent test team can provide much greater value to the organization beyond the simple finding bug job.
But, personally, I think even many experienced testers have yet to realize their full potential or the significant impact the testing role can provide beyond finding bugs. But, I do think within the next few years we will see a transition in the industry away from hiring a plethora of “expert bug-finders” towards the maturation of a respected discipline of professional software testers.