Why computer science knowledge helps testers: Example #1

There are many people who think Microsoft (and other companies, for that matter) hire programmers to be testers so they can write lot’s of automation. This, of course is not the case – we hire people who understand more about how computers work because they are better testers.

I have a handful of examples I share on this subject, so I thought I’d share a few of them here from time to time.

Let’s say you are a tester, and have two input fields in one of the dialogs of your application:

What do you test? Go ahead – make a mental list of your tests.

Really – go ahead…

Fine – I’m sure you thought of things like non-numeric numbers, large numbers, negative numbers, etc. You may have specifically tried to figure out what the maximum allowed value was, or you may have tested this inadvertently while trying to figure out the largest number possible.

But something strange happened when you tested negative numbers:

Oops: this set of inputs generated an out of memory error!

How many of you immediately inferred that the two numbers must be used for a memory allocation, and that the signed values were probably being interpreted as large unsigned numbers?

Let’s test the theory and try a variation:

No error – but this input generated output with a height of 10, and an extremely long width. We’ve confirmed that the signed input is being treated as a large value, but we’re not done yet. What happens when we launch the same dialog again?

That’s strange…or is it? Recognize that number? It looks like the first several digits of the max value of a 32 bit unsigned integer.

I haven’t looked at the code or attached a debugger or done anything other than basic black box testing, but now I can pretty much tell you exactly how the underlying code is implemented. If I have other similar dialogs in this application, I can look for the same errors there. I can tell the developer exactly what the problem is, and he can fix it quickly – and address all the problems at once, saving time, and of course, money.

Unfortunately, I know that many testers would just report that the dialog “didn’t work right”. Most would also probably report these issues as several bugs, and may not even note the interesting values, or bother to try to infer anything about how the application was working.

This in turn, causes the bug to be passed back and forth from developer to tester until everything “looks right” (hopefully the customer will see the same thing too). But that’s a waste of time. Enough of a waste of time that I think it makes much more sense to just hire testers who understand how computers work, and then pay them just as much money as their dev counterparts. The money saved in the long run more than pays for the salary costs.

Or, you can hire the boss’s nephew and his friends for the weekend to test your product.

Your call.