Tuesday, August 26, 2008

Mocks vs Traditional Asserts

I encountered something extremely interesting regarding assertions versus mocks. Traditional assertions (e.g. the NUnit asserts) are typically for "state-based" type testing while mocks are for "interaction-based" type testing.

I find that I am using assertions less and less. I'm not sure if that is a good or bad thing. A few weeks ago, I was working on adding some functionality to one of the domain objects for the project at work which unlike for Controller classes in MVC tend to be more state based testing than interaction based. At least that's what I thought.

Below is a simple method that I needed to test:
// domain object- TaskManager
public IList<Task> Reassign(IList<Task> tasks, string newTeam)
{
foreach (Task task in tasks)
{
task.Team = newTeam;
}

return tasks;
}
The following test uses traditional assertions:
 [Test]
public void CanReassignTasksToNewTeamWithAsserts()
{
TaskManager manager = new TaskManager();

const string oldTeam = "Old Team";
const string newTeam = "New Team";

// setup test data for tasks
Task task;
for (int idx = 0; idx < 3; idx++)
{
task = new Task();
task.Team = oldTeam;
tasks.Add(task);
}

// assert the re-assignment
IList<Task> updatedTasks = manager.Reassign(tasks, newTeam);
foreach (Task updatedTask in updatedTasks)
{
Assert.That(updatedTask.Team, Is.EqualTo(newTeam), "The task's team was not re-assigned.");
}
}

Now here is another test whose intention is to test the exact same thing but using mocks instead:


[Test]
public void CanReassignTasksToNewTeamWithMocks()
{
TaskManager manager = new TaskManager();

const string newTeam = "New Team";

// setup test data for tasks and set expectations
Task task ;
for (int idx = 0; idx < 3; idx++)
{
task = Mocks.CreateMock<Task>();
tasks.Add(task);

// set expectation to assign task to new team
task.Team = newTeam;
LastCall.Repeat.Once();
}

Mocks.ReplayAll();

manager.Reassign(tasks, newTeam);

Mocks.VerifyAll();
}

Guess which one I wrote first? Of course the one with mocks even though I started with the complete intention of doing it with state-based assertions but it quickly morphed to using mocks.

It was really interesting to produce these two tests that are functionally different but accomplish the same goal. They both "fail" if you remove the line:

task.Team = newTeam;

or if you place a different value:

task.Team = "Make this test fail.";

Since they fail by doing either of the above that means both tests are good, valid tests, right?

So, which should I use? Truthfully, the test with mocks is less brittle because you do not need a real instance of the "Task" object. But am I taking it too far? One "problem" I seem to have is that since I have been using mocks for so long my mind is wired to use them for everything (once again, is that a good thing or an anti-pattern?) Basically, when I think about how to test something I immediately think in terms of expectations with dependencies.

Perhaps mocks win out in this situation and in most it is better because true unit testing means that the only real instance of an object is the one that you are trying to test and essentially everything else should be mocked and/or stubbed somehow. Perhaps assertions are best with objects that are not primarily defined by their dependencies and that simply perform complex algorithms that return value types results (for example, static classes and methods). Of course, I could be oversimplifying that but I find it really hard to know when to use plain vanilla assertions.

Well, shortly after stumbling upon this "dilemma" on my own, I then read Martin Fowler's article named Mocks Aren't Stubs and it became much clearer to me what I was doing and why (as it always seem to happen whenever I read any of Fowler's stuff). According to him, I would be classified as a "mockist TDD practitioner".

Honestly, some of the reasons he lists for choosing not to be one (as opposed to a "classical TDD practitioner") are things that I definitely felt on my own especially recently when I was struggling with writing a bunch of mock heavy tests that started to get unwieldy and far more complex than the thing I was actually testing (let's just say it was a weird, dark period in my recent dev efforts that I was really questioning the use of mocks).

The quote below from him is definitely something that I thought to myself off and on for as long as I have been doing "mock testing":

"...A mockist is constantly thinking about how the SUT ["system under test" a.k.a. the object under test] is going to be implemented in order to write the expectations. This feels really unnatural to me..."

2 comments:

Unknown said...

I try to save my mock objects for the seams between layers, even then I try to use a stub first.

If I have several classes in a particular unit test, then fine. No one said that we have to have one test class per class, your SUT may be made up of several classes. See Kent Beck and Fowler.

I guess my definition of unit tests has changed over the years.

Ray Vega said...

@sneal

Not unlike yourself, since I originally wrote this I now primarily follow more the "classical" state based type testing and try to restrict mocking to external dependencies (e.g. databases, services, etc.) whenever I can.