How would you prove to a skeptic that you can count to a million (and pronounce each of the numbers)? Proving you can count to 10 is easy since you can just rattle it off in a few seconds. This is the same sort of daily dilemma a tester faces: how do you prove something when you can’t just churn every possible combo. Ultimately, you need to make some assumptions, and so you can view the real problem as “what assumptions are you willing to make”.
A few approaches I’ve seen so far:
- Spot checking:
a) Have somebody count to 100 and then assume they can count to a million. The assumption here is that demonstrating basic proficiency is sufficient.
b) Counting to 100 is quick but doesn’t test the ability to hit large numbers. An alternative approach is to count in a range from (17 – 50 by ones), (1,019 – 1,102 by twelves), etc.
- Solving a related problem. If there’s a very high probability that solving problem A means you can solve problem B, and proving A is very cheap; then proving A may be the cheapest way to increase the confidence that you can solve B.
a) Ask somebody to pronounce several random numbers (1,768; 548,842; etc), and assume that they could fill in the gaps.
b) Have somebody do some math problems like (457,233+3,917=?) . .
- Decomposition. If problem B is really just a huge combination of a few small problems A1, A2, A3…, then prove you can solve all the small problems and that you can compose them. If you put great thought into decomposition, you can get probably get a small set that tests a very large area.
a) Calculate a set of numbers that exercise every sub-word used in the words between one and a million, and then pronounce those.