Interview Tip: Writing test cases

  • Article

If during a dev interview, you're asked to write a set of test cases for some function you wrote, be sure to have test cases that reach every basic block. In other words, make sure your test cases at least hit every line of code you wrote.

For example, if you need to write 'bool IsEven(int x)' which returns true iff x is even, and you write something like this:

1: bool IsEven(int x)
2: {
3:    if (x & 1)
4:        return false; // odd
5:    else
6:        return false;  // even:   bug here!! Should return true
7: }

It's bad enough that you have a bug in line 6.

But then if the followup interview question is "Write some test cases for your function", and your test cases are:
    assert(IsEven(7) == false)
    assert(IsEven(3) == false)

You're doubley doomed.

Other quick rules for test cases. At least ensure your test cases:

  1. Hit every line of code you wrote
  2. Test corner cases such as  0, null, recursion base cases, large values, etc
  3. expose bugs in alternative buggy implementations. For example, if you choose not to use implementation X because it has bug Y, then include a test case that hits bug Y.
  4. hit Error cases if applicable.

 

I think some good test cases for IsEven above would be:

assert(IsEven(0) == true)   // 0 is a common corner case
assert(IsEven(-1) == false) // negatives are a good corner case.
assert(IsEven(-2) == true) // test negative  numbers
assert(IsEven(5) == false)  //
assert(IsEven(18) == true)  //
assert(IsEven(3000000) == true) // A recursive implementation would probably hit a stack overflow here.
assert(IsEven(3000001) == false) // A recursive implementation would probably hit a stack overflow here

Bonus points: more complex test cases.
The tests above are all "baseline tests" that take an input and match it against an expected output. This has the challenge of manually picking good numbers for the baseline. 

If the baseline tests are all the interviewer is interested in, then you're good with the above. If that satisfies the interview, great! Move on to the next question. (related Interview tip: don't wast the interviewer's time)

But if you're asked something like "how else could you test this to be super confident that it's really correct in all case", you could have some fun with more complex tests that use properties / identities of your function.  For example, you know that:
    assert(IsEven(x) == !IsEven(x+1))

Thus you could write a test that iterates through all possible inputs and verifies it against all outputs. Toss in that your function is stateless and now your talking about empirically provably correctness. That's sure to get some bonus points in an interview. :)