Tuesday, September 9, 2008

Cryptic Rhino Mock exception messages

Let me first start off by saying that Rhino Mocks is a great mock objects framework for unit testing in .NET and C#. As compared with NMock2, which was my first experience with testing using mock objects, it is far superior (the use of strongly typed method/property names instead of strings is one of its best features especially for TDD and refactoring.) However, there are some aspects of NMock2 that I do miss.

'Expect' Consistency

For starters, NMock2 was more consistent in how the 'Expect' calls are made versus the way Rhino Mocks does it. In NMock2, the use of 'Expects' are the same whether you use a void method or a method that returns a value:
Expect.Once.On(mockFoo).Method("SomeMethodThatReturnsAValue")
Expect.Once.On(mockFoo).Method("SomeVoidMethod")
That is not the case with Rhino Mocks. 'Expects' can only be used with methods that return values and not with void methods.

Recently, a new way of expressing 'Expects' with void methods was added to the Rhino Mocks framework but it relies on 'delegates'. Not sure if I really like the solution. It trades off one form of weak readability for another albeit different one.

This could be yet another reason to turn off newbies from testing with a mock framework such as Rhinos. It can be confusing. It is already quite a difficult endeavor to encourage software developers the virtues of unit testing. It is even more difficult to promote mock object testing so anything to lower the barriers is important and critical.

Understandable Exception Messages

In addition, Rhino Mock exception messages sometimes can be vague and unclear. This can be frustrating for new (and even existing) users.

For example, I was recently working on an old test fixture for a project which uses NMock2 and not Rhinos as its testing framework. To some degree, I felt a bit more productive with and in control of it because the error messaging is a lot more user friendly. I could more quickly determine the cause of a problem.

For example, below is an actual exception I received from NMock2:

NMock2.Internal.ExpectationException: not all expected invocations
were performed
Expected:
1 time: criteria.SetFirstResult(equal to <50>) [called 0 times]
1 time: criteria.SetMaxResults(equal to <5>) [called 0 times]
1 time: criteria.List(any arguments), will return
<System.Collections.Generic.List`1[System.DateTime]> [called 0 times]

Now, here is what I might get from Rhinos:
Rhino.Mocks.Exceptions.ExpectationViolationException:
ICriteria.SetFirstResult(50); Expected #1, Actual #0.
ICriteria.SetMaxResults(5); Expected #1, Actual #0.

Honestly, I like the first one better. It reads better to me. For one thing, Rhinos provides the raw (CLR?) object definition so that if the member is inherited from an interface or another class then it shows as it is defined for interface (i.e. "ICriteria") or the base class . Meanwhile, NMock2 shows the actual local variable name used in the code you are testing (i.e. "criteria"). Much faster to pinpoint the culprit.

In fact, where this really drives me crazy is for the domain objects (i.e. POCOs, business objects, etc.) for that same project. Every domain object inherits from IDomainObject so with Rhino Mocks I get this:
IDomainObject.Description
OK....but, which domain object is it? If I happen to have two or more domain objects being mocked/stubbed in my test it can get really hard figuring out the one it's complaining about. Instead, it would be nice if Rhino provided the following as does NMock2 using the variable name (assuming my domain object is named 'Foo'):
foo.Description
Another example of the disparity between the two frameworks is if a property related exception occurs then the Rhino message would contain this:
IFooView.set_PageSize
while NMock2 would provide this:
_view.PageSize \\ instance variable name
Some would say, "What's the big deal?", "Can't you figure out what it is?", "It only takes a few seconds to know what it is", etc. Well, that is the problem. If my brain has to stop to process what it is, even if it takes a few seconds, then that is slowing me down during my software development process. Multiply those "few" seconds by how many times you get Rhino exceptions like that and it does eat away at your development time. It does add up over time. It is not unlike trying to read code that is not very readable or well factored. Sure, you'll eventually figure out what it does but at the cost of precious dev time.

The following exception message is one I'm fairly certain I have gotten before but always forget because the message is so...well...CRYPTIC!!!!
System.InvalidOperationException: Previous method 'IView.get_ReturnSomeStringValue();' require a return value or an exception to throw.
If you specify the wrong data type in the 'Return' method of an Expect (or LastCall) then the above exception will be thrown. For example, if the method or property is suppose to return a 'string' type value but you instead specify a 'DateTime' type as shown below:

DateTime date = DateTime.Today
Expect.Call(_view.ReturnSomeStringValue).Return(date);
// this will throw an exception
then you will receive the error message mentioned earlier.

Specifying the proper type should fix the problem as follows:

string someStringValue = "some string value";
Expect.Call(_view.ReturnSomeStringValue).Return(someStringValue);
// this is ok
The exception message should really be about checking for strong typing and not the absence or lack of a return value.

6 comments:

Anonymous said...

I agree, Rhino Mock and MoQ as well, are strongly typed. But we invest a lot to provide good exception messages and a clear syntax with NMock2. That's the reason we stick with NMock2: clear and easy syntax, good exception behavior and good extensibility.

Happy mocking (which ever mocking framework you use :-)

Ray Vega said...

@urs enzler

I found if you use ReSharper then the strings as used in NMock2 can just as easily be renamed as if they were strongly typed.

The only catch is Re# is not free. Not an issue for me since I have a license but can be for others.

Unknown said...

Overall I prefer the Rhino error messages since they are short and concise, however I do agree:

1. Property errors should show the C# name, not the underlying .NET name. set_MyProp is a little annoying.
2. I like the consistency of Expect in NMock2, this is especially true for noobs.

I'm pretty sure your last example (string with DateTime) won't compile in RhinoMocks 3.4 since it uses generics. At one point it didn't.

RhinoMocks 3.5 syntax is much cleaner and more consistent IMO. So drop everything else and use that.

Ray Vega said...

@sneal

Admittedly, I wrote this over a *year* ago so it was based on whatever version Rhino Mocks was at the time.

Haven't had an opportunity lately to see if it's improved but good to know.

Anonymous said...

Ray, as you say the article is quite old but nothing seem to have changed in error messages by now :)

Just starting to fumble with RhinoMock - C#3 support in the latest version really makes a difference.

BTW, Expects still don't work with void methods - but lambda expressions make this problem less painful even in "old-fashioned" way of setting expectations:

Expect.Call(() => hand.TouchHotIron(iron)).Throw(new BurnException());



(just a note to someone who may have googled the article)

Ray Vega said...

Andrew

I have not been fortunate enough to work on a project using the latest in Rhino Mocks coupled with C# 3.x but I definitely would like to experience for myself some of the syntactical improvements you mention.

I do plan on taking a look at Moq (http://code.google.com/p/moq/) to see how it holds up in those same areas.