A Tester's Translation Table

  • Article

It seems to me that lots of people are experiencing lots of confusion regarding what lots of the testing terms we throw around signify. In an effort to remedy this circumstance I have applied my investigatory powers to observe what people really mean when they say these words. Forthwith, the answers which I have compiled:

Categories of Tests

  • Build Verification Tests: "Did they even bother to test this?" The oft-used synonym "smoke test" comes from the similar question "What were they smoking when they checked this in?"
  • Exit Scenario Tests: A test of the executives' exit strategies.
  • Basic Functionality Tests: The set of all tests which are executed.
  • Comprehensive Tests: The set of tests which will never ever be executed.

Build Ratings

  • Self Toast: The build had a nasty bug and now the lab machines are all being rebuilt.
  • Self Test: The build had a nasty bug and now the product team's machines are all being rebuilt.
  • Self Host: The build had a nasty bug and now the executives' machines are all being rebuilt.

Types of Tests/Testing

  • Positive Test: A test which is so confident it will never fail that it does not bother to verify and/or log anything.
  • Negative Test: A test which has never ever passed. And no one cares.
  • Black Box Testing: Blindly flailing about hoping against hope to find a bug.
  • White Box Testing: Coding.
  • User Scenario Testing: Pretending we understand how our customers are going to use our product.
  • Acceptance Testing: Pretending we know how to tell whether a feature is done.
  • Regression Testing: Verifying that the developers didn't really fix those bugs.
  • Performance Testing: Annual exercise to determine how big of a bonus each executive receives.
  • Load Testing: Determining how much overtime the team can take before they fall ill.
  • Exploratory Testing: "Hey tester, go do whatever you want and tell me when you're done."
  • Scripted Testing: "Hey tester, exactly execute these test steps I wrote ages ago, back before anybody knew anything about how anything would actually work."

Bug Resolutions

  • By Design: That "bug" is really a feature.
  • Duplicate: A quick way to raise your score in bug bashes.
  • External: Not our problem.
  • Fixed: It works on my machine!
  • No Repro: I don't want to deal with this.
  • Postponed: Let's all pretend we will fix this some day.
  • Won't Fix: As if! Why did you even bother logging this?

(Yes, my tongue is firmly in my cheek! To learn what I really think these terms mean, see my DDJ posts Test Taxonomy Decoder Ring and More Decoder Rings.)

*** Do these definitions seem all too familiar? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.