It seems to me that lots of people are experiencing lots of confusion regarding what lots of the testing terms we throw around signify. In an effort to remedy this circumstance I have applied my investigatory powers to observe what people really mean when they say these words. Forthwith, the answers which I have compiled:
Categories of Tests
- Build Verification Tests: “Did they even bother to test this?” The oft-used synonym “smoke test” comes from the similar question “What were they smoking when they checked this in?”
- Exit Scenario Tests: A test of the executives’ exit strategies.
- Basic Functionality Tests: The set of all tests which are executed.
- Comprehensive Tests: The set of tests which will never ever be executed.
- Self Toast: The build had a nasty bug and now the lab machines are all being rebuilt.
- Self Test: The build had a nasty bug and now the product team’s machines are all being rebuilt.
- Self Host: The build had a nasty bug and now the executives’ machines are all being rebuilt.
Types of Tests/Testing
- Positive Test: A test which is so confident it will never fail that it does not bother to verify and/or log anything.
- Negative Test: A test which has never ever passed. And no one cares.
- Black Box Testing: Blindly flailing about hoping against hope to find a bug.
- White Box Testing: Coding.
- User Scenario Testing: Pretending we understand how our customers are going to use our product.
- Acceptance Testing: Pretending we know how to tell whether a feature is done.
- Regression Testing: Verifying that the developers didn’t really fix those bugs.
- Performance Testing: Annual exercise to determine how big of a bonus each executive receives.
- Load Testing: Determining how much overtime the team can take before they fall ill.
- Exploratory Testing: “Hey tester, go do whatever you want and tell me when you’re done.”
- Scripted Testing: “Hey tester, exactly execute these test steps I wrote ages ago, back before anybody knew anything about how anything would actually work.”
- By Design: That “bug” is really a feature.
- Duplicate: A quick way to raise your score in bug bashes.
- External: Not our problem.
- Fixed: It works on my machine!
- No Repro: I don’t want to deal with this.
- Postponed: Let’s all pretend we will fix this some day.
- Won’t Fix: As if! Why did you even bother logging this?
*** Do these definitions seem all too familiar? I need a tester! Interested? Let’s talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.