Cryptic Rhino Mock exception messages

Let me first start off by saying that Rhino Mocks is a great mock objects framework for unit testing in .NET and C#. As compared with NMock2, which was my first experience with testing using mock objects, it is far superior (the use of strongly typed method/property names instead of strings is one of its best features especially for TDD and refactoring.) However, there are some aspects of NMock2 that I do miss.

'Expect' Consistency

For starters, NMock2 was more consistent in how the 'Expect' calls are made versus the way Rhino Mocks does it. In NMock2, the use of 'Expects' are the same whether you use a void method or a method that returns a value:

Expect.Once.On(mockFoo).Method("SomeMethodThatReturnsAValue")
Expect.Once.On(mockFoo).Method("SomeVoidMethod")

That is not the case with Rhino Mocks. 'Expects' can only be used with methods that return values and not with void methods.

Recently, a new way of expressing 'Expects' with void methods was added to the Rhino Mocks framework but it relies on 'delegates'. Not sure if I really like the solution. It trades off one form of weak readability for another albeit different one.

This could be yet another reason to turn off newbies from testing with a mock framework such as Rhinos. It can be confusing. It is already quite a difficult endeavor to encourage software developers the virtues of unit testing. It is even more difficult to promote mock object testing so anything to lower the barriers is important and critical.

Understandable Exception Messages

In addition, Rhino Mock exception messages sometimes can be vague and unclear. This can be frustrating for new (and even existing) users.

For example, I was recently working on an old test fixture for a project which uses NMock2 and not Rhinos as its testing framework. To some degree, I felt a bit more productive with and in control of it because the error messaging is a lot more user friendly. I could more quickly determine the cause of a problem.

For example, below is an actual exception I received from NMock2:

NMock2.Internal.ExpectationException: not all expected invocations
were performed
Expected:
1 time: criteria.SetFirstResult(equal to <50>) [called 0 times]
1 time: criteria.SetMaxResults(equal to <5>) [called 0 times]
1 time: criteria.List(any arguments), will return
<System.Collections.Generic.List`1[System.DateTime]> [called 0 times]

Now, here is what I might get from Rhinos:

Rhino.Mocks.Exceptions.ExpectationViolationException:
ICriteria.SetFirstResult(50); Expected #1, Actual #0.
ICriteria.SetMaxResults(5); Expected #1, Actual #0.

Honestly, I like the first one better. It reads better to me. For one thing, Rhinos provides the raw (CLR?) object definition so that if the member is inherited from an interface or another class then it shows as it is defined for interface (i.e. "ICriteria") or the base class . Meanwhile, NMock2 shows the actual local variable name used in the code you are testing (i.e. "criteria"). Much faster to pinpoint the culprit.

In fact, where this really drives me crazy is for the domain objects (i.e. POCOs, business objects, etc.) for that same project. Every domain object inherits from IDomainObject so with Rhino Mocks I get this:

IDomainObject.Description

OK....but, which domain object is it? If I happen to have two or more domain objects being mocked/stubbed in my test it can get really hard figuring out the one it's complaining about. Instead, it would be nice if Rhino provided the following as does NMock2 using the variable name (assuming my domain object is named 'Foo'):

foo.Description

Another example of the disparity between the two frameworks is if a property related exception occurs then the Rhino message would contain this:

IFooView.set_PageSize

while NMock2 would provide this:

_view.PageSize \\ instance variable name

Some would say, "What's the big deal?", "Can't you figure out what it is?", "It only takes a few seconds to know what it is", etc. Well, that is the problem. If my brain has to stop to process what it is, even if it takes a few seconds, then that is slowing me down during my software development process. Multiply those "few" seconds by how many times you get Rhino exceptions like that and it does eat away at your development time. It does add up over time. It is not unlike trying to read code that is not very readable or well factored. Sure, you'll eventually figure out what it does but at the cost of precious dev time.

The following exception message is one I'm fairly certain I have gotten before but always forget because the message is so...well...CRYPTIC!!!!

System.InvalidOperationException: Previous method 'IView.get_ReturnSomeStringValue();' require a return value or an exception to throw.

If you specify the wrong data type in the 'Return' method of an Expect (or LastCall) then the above exception will be thrown. For example, if the method or property is suppose to return a 'string' type value but you instead specify a 'DateTime' type as shown below:

DateTime date = DateTime.Today
Expect.Call(_view.ReturnSomeStringValue).Return(date);
// this will throw an exception

then you will receive the error message mentioned earlier.

Specifying the proper type should fix the problem as follows:

string someStringValue = "some string value";
Expect.Call(_view.ReturnSomeStringValue).Return(someStringValue);
// this is ok

The exception message should really be about checking for strong typing and not the absence or lack of a return value.

Mocks vs Traditional Asserts

I encountered something extremely interesting regarding assertions versus mocks. Traditional assertions (e.g. the NUnit asserts) are typically for "state-based" type testing while mocks are for "interaction-based" type testing.

I find that I am using assertions less and less. I'm not sure if that is a good or bad thing. A few weeks ago, I was working on adding some functionality to one of the domain objects for the project at work which unlike for Controller classes in MVC tend to be more state based testing than interaction based. At least that's what I thought.

Below is a simple method that I needed to test:

// domain object- TaskManager
public IList<Task> Reassign(IList<Task> tasks, string newTeam)
     {
         foreach (Task task in tasks)
         {
            task.Team = newTeam;
         }

         return tasks;
     }

The following test uses traditional assertions:

 [Test]
     public void CanReassignTasksToNewTeamWithAsserts()
     {
         TaskManager manager = new TaskManager();
     
         const string oldTeam = "Old Team";
         const string newTeam = "New Team";

         // setup test data for tasks
         Task task;
         for (int idx = 0; idx < 3; idx++)
         {
             task = new Task();
             task.Team = oldTeam;
             tasks.Add(task);
         }

         // assert the re-assignment
         IList<Task> updatedTasks = manager.Reassign(tasks, newTeam);
         foreach (Task updatedTask in updatedTasks)
         {
             Assert.That(updatedTask.Team, Is.EqualTo(newTeam), "The task's team was not re-assigned.");     
         }
     }

Now here is another test whose intention is to test the exact same thing but using mocks instead:

[Test]
     public void CanReassignTasksToNewTeamWithMocks()
     {
         TaskManager manager = new TaskManager();

         const string newTeam = "New Team";

         // setup test data for tasks and set expectations
         Task task ;
         for (int idx = 0; idx < 3; idx++)
         {
             task = Mocks.CreateMock<Task>();
             tasks.Add(task);

             // set expectation to assign task to new team
             task.Team = newTeam;
             LastCall.Repeat.Once();
         }

         Mocks.ReplayAll();

         manager.Reassign(tasks, newTeam);

        Mocks.VerifyAll();
     }

Guess which one I wrote first? Of course the one with mocks even though I started with the complete intention of doing it with state-based assertions but it quickly morphed to using mocks.

It was really interesting to produce these two tests that are functionally different but accomplish the same goal. They both "fail" if you remove the line:

task.Team = newTeam;

or if you place a different value:

task.Team = "Make this test fail.";

Since they fail by doing either of the above that means both tests are good, valid tests, right?

So, which should I use? Truthfully, the test with mocks is less brittle because you do not need a real instance of the "Task" object. But am I taking it too far? One "problem" I seem to have is that since I have been using mocks for so long my mind is wired to use them for everything (once again, is that a good thing or an anti-pattern?) Basically, when I think about how to test something I immediately think in terms of expectations with dependencies.

Perhaps mocks win out in this situation and in most it is better because true unit testing means that the only real instance of an object is the one that you are trying to test and essentially everything else should be mocked and/or stubbed somehow. Perhaps assertions are best with objects that are not primarily defined by their dependencies and that simply perform complex algorithms that return value types results (for example, static classes and methods). Of course, I could be oversimplifying that but I find it really hard to know when to use plain vanilla assertions.

Well, shortly after stumbling upon this "dilemma" on my own, I then read Martin Fowler's article named Mocks Aren't Stubs and it became much clearer to me what I was doing and why (as it always seem to happen whenever I read any of Fowler's stuff). According to him, I would be classified as a "mockist TDD practitioner".

Honestly, some of the reasons he lists for choosing not to be one (as opposed to a "classical TDD practitioner") are things that I definitely felt on my own especially recently when I was struggling with writing a bunch of mock heavy tests that started to get unwieldy and far more complex than the thing I was actually testing (let's just say it was a weird, dark period in my recent dev efforts that I was really questioning the use of mocks).

The quote below from him is definitely something that I thought to myself off and on for as long as I have been doing "mock testing":

"...A mockist is constantly thinking about how the SUT ["system under test" a.k.a. the object under test] is going to be implemented in order to write the expectations. This feels really unnatural to me..."