The mess that mocks can make


Aside: I guess this post is really about mock frameworks rather than mocks, but I didn't want to break the partial alliteration I had going in the title 🙂

A question was posed recently on one of our discussion lists about whether Rhino Mocks was a good framework to use for unit tests. As a TDD EDD advocate, conversations like that fire off warning bells as for the most part, I don't believe that they are suited to writing unit tests in the context of test-first development. I'd go so far as to say that they can even be detrimental.

As an example, here are a couple tests from the Humble Dialog Box post which is part of Jeremy Miller's series on 'Building your own CAB'.

[Test]
public void CloseTheScreenWhenTheScreenIsNotDirty()
{
MockRepository mocks = new MockRepository();
IHumbleView view = mocks.CreateMock();

Expect.Call(View.IsDirty()).Return(false);
view.Close();

mocks.ReplayAll();

OverseerPresenter presenter = new OverseerPresenter(view);
presenter.Close();

mocks.VerifyAll();
}

[Test]
public void CloseTheScreenWhenTheScreenIsDirtyAndTheUserDecidesTo
DiscardTheChanges()
{
MockRepository mocks = new MockRepository();

IHumbleView view = mocks.CreateMock();

Expect.Call(view.IsDirty()).Return(true);
Expect.Call(view.AskUserToDiscardChanges()).Return(true);
view.Close();

mocks.ReplayAll();

OverseerPresenter presenter = new OverseerPresenter(view);
presenter.Close();

mocks.VerifyAll();
}

A few questions spring to mind:

What story does each test tell? The method names are very descriptive, but the code doesn't provide clear support for the behaviour.

Where have the three As disappeared to? What happened to the expected trilogy of sections: Arrange, Act, and Assert?

and perhaps most importantly,

What happens when I want to refactor?

If you look carefully at how the mock object is created, you're really just setting up (a) expected behaviours and possibly (b) an expected sequence of calls. When code is refactored, unless you're accessing a public API, chances are that you will need to change both (a) and (b). That means going back through each unit test and making the changes by hand. Not very agile.

Furthermore, if you're using NMock and you want to change the interface (IHumbleView in the example), you'll also need to go back and change all the method names by hand since that framework uses strings. Also not very agile. In both cases, this leads people to abstain from refactoring, which is one of the major benefits of practicing test-first development.

There are two other points as well:

  • Giving a brand-new test-first developer a mock framework is probably hazardous to their health. It can easily lead to poor habits such as writing tests that are highly coupled to the implementation of the classes being tested. See (b) above.
  • Except for very small fixtures, there is actually less code involved in writing a real mock object. Why would anyone want to make extra work for themselves?

Now compare the above tests to these ones that were written without a mock framework.

[Test]

public void ViewClosesWhenTheScreenIsNotDirty()

{

    MockHumbleView view = new MockHumbleView();

    Presenter presenter = new Presenter(view);

 

    presenter.Close();

 

    Assert.IsTrue(view.CloseWasCalled);

}

 

[Test]

public void ViewClosesWhenScreenIsDirtyAndUserDiscardsChanges()

{

    MockHumbleView view = new MockHumbleView();

    view.DiscardChanges = true;

    Presenter presenter = new Presenter(view);

 

    presenter.Close();

 

    Assert.IsTrue(view.CloseWasCalled);

}

and the matching code ...

public interface IHumbleView

{

    bool IsDirty { get; }

    bool DiscardChanges { get; }

    void Close();

}

 

public class MockHumbleView : IHumbleView

{

    public bool IsDirty

    {

        get { return isDirty; }

        set { isDirty = value; }

    }

 

    public bool DiscardChanges

    {

        get { return dischardChanges; }

        set { dischardChanges = value; }

    }

 

    public bool CloseWasCalled

    {

        get { return closeWasCalled; }

    }

 

    public void Close()

    {

        closeWasCalled = true;

    }

 

    private bool isDirty;

    private bool dischardChanges;

    private bool closeWasCalled;

}

To be honest I'd take it one step further and actually test everything directly through the view but that's not what this post is about.

I think everyone can agree that the second group of tests is much easier to read and definitely convey what they're trying to prove. If I was introducing someone to the concept of test-first development, I'd want them to start writing tests like that for a while before worrying about mock object frameworks.

Mock frameworks are probably useful in some situations, but think carefully if you really require them in a before diving in.

Update: Steve Otteson pointed out (rather correctly) that I'm really talking about 'stubs' when I refer to 'real mock objects'. The difference between the two is explained here.

Comments (8)
  1. Micah says:

    I’m actually new to agile development, and I was using Rhino Mock to test a controller by mocking a passive view when I realized I was not gaining anything. It took about five minutes to write a mock that wouldn’t break when the controller was refactored.

    It seems that mock frameworks are something that I will know when I need, but until then they should be avoided.

  2. Casper says:

    Yes, it’s too bad these frameworks don’t come with a warning sticker on them 🙂 I wouldn’t say it’s necessarily something you’ll ever need either. You may, and it’s good to know they exist, but so far I’ve been quite happy without them.

  3. Well, I disagree with you in couple of things 🙂

    Regarding a) “the behavior”

    By my understanding, the purpose of mocking is to test the behaviors VS unit tests which test the execution contexts. If I would right the test with some mock about some method, I am testing and contracting the behavior of that method. If that would change, I would like to have a broken test pointing me to that. In case I wouldn’t want to be constrained with contracted behaviors, I wouldn’t use mocking framework at all

    Regarding b) “the sequence”

    Rhino mocks recording mode is by default set to unordered, so in this example even if you change the sequence unit test would still work. If you manually set the mockery to record expectations in ordered mode, where sequence is important, I think that represent the test intention which I would like, in case sequence had changed, to result in broken test

    Regarding stub vs mock

    TDD (EDD) as “test first” development environment requires abstracted “fake” representations of the concepts tested even before the real one would be created. First downside of using stubs is that at the end of big project there’s a big pile of type stubs totally unused which complete removal is usually very tricky and time consuming task. Second downside is that stubs are not automatically updated when the type they stub would change. Therefore the test would still be green with the stub, but the stub would be wrong

  4. Brian Button says:

    Ah, Casper,

    You have channeled my question about Mocks and Mock Object Frameworks 100%. I have the exact same questions as you, the exact same reactions as you, and I’m just as confused as you.

    I tracked down a few different people at the conference to have them explain it to me and it makes a bit more sense. If you look at the blog entry I pointed you to at the conference (A Deep Dive into Test Driven Development, Oct, 2004), I’m planning on redoing that example using Rhino Mocks and blogging where it takes me.

    There is enough refactoring in there to test out this way of writing tests. Who knows, maybe I’ll learn something new!

    — bab

  5. Casper says:

    Looking forward to reading about your discoveries Brian 🙂 I did read through your Deep Dive series, as well as the 2005 refactoring of the Video Store example.

    I hate to be an echo chamber, but I did learn a few things from those posts and so I’ll be linking to them soon 🙂

  6. casper says:

    Nikola: Sorry for taking so long to reply.

    a) I’m not claiming that mocks don’t test the contract or behaviour of a method. My point is that you often write several tests for a single method, and if you are using a mock framework, it’s not easy to refactor that method if/when it changes. Your tests will still fail if you create a real class for the mock.

    b) I don’t think I’ve claimed that order of the tests is important anywhere .. maybe you could explain this point a bit more?

    c) I’ll have to disagree that having a ‘big pile of type stubs’ at the end of a project is a bad thing. They should be located in your test assembly and not have any effect on the rest of the system. I’m also not sure why you would want to completely remove them.

  7. malovicn says:

    I was commenting your next sentences:

    ========

    If you look carefully at how the mock object is created, you’re really just setting up (a) expected behaviours and possibly (b) an expected sequence of calls.  When code is refactored, unless you’re accessing a public API, chances are that you will need to change both (a) and (b). That means going back through each unit test and making the changes by hand. Not very agile

    ===============

    re: a)

    The difference you are describing exists only in NMock where method names are strings. My point was that with the Rhino Mocks we have strongly typed mock members accessors so there is no difference at all (in that sense) in refactoring benefits of using stubs vs mocks  

    re: b)

    I just said that if you don’t explicitly contract the "expected sequence of calls" (that stands for ordered set of action, right?) Rhino mocks won’t break a test.By default, Rhino mock don’t care about the order of expectations.

    re: c)

    I don’t see any advantage in using stubs in tests (two points above are some of examples), so beeing forced to create a bunch of "paralel" types somewhere just to run tests sounds as an extra work to me. Another thing, in case of stubs when "original" type is been updated, the "stubbed" one has to be updated manualy too. In case of mocks (due to their dynamic nature) no extra "parallel" work needed

  8. malovicn says:

    Just a side note (delete it if you like): I don’t know why I’ve now different nick on the site but malovicn and Nikola Malovic are both me 🙂

Comments are closed.

Skip to main content