To mock or not to mock?

  • Article

In Wikipedia's definition of Unit test, it says "Ideally, each test case is independent from the others; mock or fake objects as well as test harnesses can be used to assist testing a module in isolation." If we follow this definition strictly, whenever the class under test collaborates with other classes, we need to mock the dependent classes. (Here we do not differentiate among mock, stub, fake, etc. A test double might be a better term.) The main benefit of mocking is that we can test our class in isolation. As a result,

  1. If the dependent classes are hard to set up, take too long to run, or are still at an early development stage, mocking can make writing tests easier, and help the tests run faster and be more stable.
  2. We can test more thoroughly by manipulating the mock objects to produce input that cannot easily be done with real objects.
  3. If a bug is introduced in one class, ideally you will only have one test case fail. We only need to look at the class under test and do not need to worry about the dependent classes. It saves test maintenance and debugging time.

The disadvantages of mocking are:

  1. It can be extra overhead. To do "pure" unit test, you either create many mock classes or have test code that are specifically for mocking if you use dynamic mocking. The test code can take longer to write and be harder to read and maintain.
  2. With mocking, unit tests are testing less. It doesn't test the potential bugs coming from the interaction between classes. Some may argue that's the point of unit testing. Collaboration between classes should be tested by integration tests or functional tests. However, unit tests are our first line of defense. If we can catch bugs early with actually less effort by not mocking, I see no reason to mock unless there are other benefits.
  3. If we are doing mocking blindly, we can miss the opportunity of using tests to make our classes more testable. For example, at an extreme, without any mocking, we can change all the interfaces to only use primitive types to achieve the goal of test in isolation. In some cases, it may indeed make sense to change the interface instead of mocking. If we always mock, it can be harder to see that.

To me, it feels more natural initially not to mock even though it means I may not be doing "pure" unit testing [1]. I'll only mock when it's necessary or it's really better than using the real objects. If we are constantly running tests (unit test or not), when tests fail, the first thing to look at is the last batch of check-ins. So the gain of mocking may not be substantial in a lot of cases. I think if our goal is to have "clean code that works," we basically want to spend minimum effort to have good design with maximum code coverage. It should be secondary whether we are doing "pure" unit tests or not.

[1] In Mocks Aren't Stubs, Martin Fowler differentiates between Classical TDDers and Mockist TDDers. Classical TDDers (Fowler being one) actually treat unit tests as mini-integration tests. I feel more comfortable with the classical TDD approach.

Keyword: unit test, mock, Martin Fowler