Sunday, 15 November 2009

Is Typemock Isolator Evil - Round N+1

Every few months the argument for and against the advance abilities of the Isolator mocking framework burst again. This time it started with this post: Test driven design – Willed vs. Forced Designs

Disclaimer: I used to work for Typemock company and was in charge for a long time over the development of the framework. so I claim no kind of objectivity .

the argument for and against usually circle around the following:

on one side, people claim that Isolator "breaks" the language barriers and, by allowing any kind of design to be implemented will end up in helping to build a poorly designed system.

on the other side, by allowing the freedom to design at will Isolator shift the responsibility back to the developer allowing him to choose the "best" design as he sees fit.

here are some points I want to comment on:

If you need Isolator you have a bad design

Actually that one is in most cases true. However the following claim "you have bad design" also holds for too many cases as well. Yes I'm saying that most systems out there are poorly designed. I'm also saying that most software project out there will fail. What I don't like about the initial claim is the conclusion that usually follows:

If you need isolator when you have poor design, usage of isolator will end up with a poor design.

That statement is plain wrong, you will end up with poor design unless you learn how to design better. The effect Isolator (or any tool for that matter) will have on a team design skills is minimal at best. Pragmatically speaking if you are are working on a legacy system isolator is probably the better choice no matter what. If you are working on a newly project and you just start out TDD, most likely using isolator will increase the chances you'll be able to stick with it. (adding the need to relearn design at this stage is soo much harder), and if you're working on a new project and you do know how to TDD. Then you should just know better and be able to safely use the tool. If you don't then you really do have a problem.

Isolator breaks the natural barriers of the language

Yes it does, but the statement "Statics are the death of testability" which actually means "don't use static methods" adds barriers to the language which is not there. so what's the better approach? again I don't know. It really depends on your personal preferences and has nothing to do with what is the best design

Actually there's no such thing is the best design. Every design must evolve always to fit the system needs.

Show me a concrete example

example number 1:

many systems have a dependency on the time of day. the naive way to approach that is to use DateTime.Now. but oops that one cant be faked (its a static method) making it a real pain to test. so experienced programmer introduce the ITime interface wrapping the system time with an interface and then adding some concrete code to make it work, all for the sake of testability. here's an example taken from the testify project on how to implement this:

public static class SystemClock {

private static DateTime? fixedTime;

public static DateTime Now {
get {
if (fixedTime.HasValue)
return fixedTime.Value;
return DateTime.Now;
}
}

internal static void Set(DateTime value) {
fixedTime = value;
}

internal static void Reset() {
fixedTime = null;
}
}
Please enlighten me as why this is better/simpler as opposed to use of DateTime.Now.


example number 2:

I have a class A depends on Class B. Class B is the ONLY concrete class implementing the interface ISomeInterafce (and most likely this will hold true forever). Class A is the only place (for now and most likely forever) ISomeInterface is used. yes this is a simple example but since I'm inventing it, I get to set the rules.

and there are several ways to approach this:

1. Use some IOC container to instantiate B and send it to A. a valid OO way with many benefits however in my concrete example (and yes I'm the one setting the rules). I don't have a true need for a full pledged IOC yet, and this very simple scenario wont be the one causing me to start using an such a big hammer

2. Use a factory - a simpler version of the IOC container idea, much more light weight. but still at this point of time (and yes I get to make the rules) under the specified circumstance it falls under the YAGNI directive. If the situation changes (i.e. more places will use ISomeInterafce or more "kinds" of ISomeInterafce will evolve) a factory will be used but for now I don't really need that.

3. Instantiate B somewhere and pass it into A (the simplest form of DI) - a very simple solution which is valid under this scenario. the down side for it are , that it exposes some inner working details of A which may not be a good idea AND most likely it only shifts the problem to someplace else.

4. Just use a new statement somewhere inside A (lets say constructor) - I like this way. Its the simplest thing that can possibly work, and I'm not tied to the "design for testability" chains, so I can do that. and just before I get chopped, yes taken outside this context of this simple concrete example is most likely not a good strategy.



I want to make the Choices. I'm the professional trained to do them. I wont stand being TOLD how to do my job, not by a person, and especially not by a tool.

9 comments:

Steve Freeman said...

Sure example 1 is disgusting, but it looks like a strawman to me, not least because it's leaking test support code into production.

The real point is that I can use the impedance of trying to test against the singleton of system time to help me think about my design.

I've seen so many systems that have been a nightmare to change because the implementation of some time-based feature was scattered across the code. It's not about hiding time behind an interface (although that may be part of the solution), but about understanding what time is being used for and modelling that in the code.

Done right, I would expect to find myself isolating all my uses of time into some relevant domain concept. I wrote this up a while ago (http://www.mockobjects.com/2007/04/test-smell-i-need-to-mock-object-i-cant.html). Nowadays I think I would push harder to cast it in terms of the domain model.

Steve Freeman said...

I want to make the Choices. I'm the professional trained to do them. I wont stand being TOLD how to do my job, not by a person, and especially not by a tool.

Too late, you're already being told by C#, the .Net libraries, other frameworks and tools, Intel, and John von Neumann. I find that even my choice of IDE colours how the code comes out.

It's not about control, it's about picking the tools that you want to work with.

Perhaps you should consider moving to lisp? :)

Lior Friedman: said...

Hi Steeve,
I think that the aim of SystemClock from example 1, is not really different from your "Clock" concept. In fact other then the actually implementation (which i don't think is the point here) its the same basic concept.
Your point, however regarding implicit dependacies is a valid one. However, taking testability out of the equation, there are some cons for "information hiding" which might outwieght the cons for exposing dependencies.

Lior Friedman: said...

Yes its too late, I was born into a limited world. and there are some degree of freedom which were taken away from me.

That's however does not encourage me to further limit myself. I'm not tieing my hands just because I can code badly.

As Roy said in the original post, I will pick the tool-set that will give me the biggest degree of freedom. And it is a personal preference only. I will risk the dangers in favor of the speed gain.

Steve Freeman said...

My experience is that using system clock in the domain model has always been problematic eventually. I think it's because doing so implicitly scatters a dependency across the code that should have been packaged up.

If you really want the greatest freedom, then surely you ought to be dropping C# for dynamic language?

Lior Friedman: said...

I think i'll wait to .net 4.0 before making that decision.

George Mauer said...

Everyone always brings up the DateTime.Now argument and I really think that its based more on how thrilled the Typemock guys are at their own cleverness (and it IS clever) than on any belief that you should not externalize a time provider.

Of course the current time should be externalized! That is something that can absolutely be in flux. Its not crazy at all to imagine that you might want at some point the system to run 2 hours ahead, round the time off to the nearest five minutes, or synchronize with an external system.

The other example is a much better one and yes, you are absolutely correct - being able to mock that out would be nice, and not necessarily poor design. However, looking back at my own projects:
a) There are usually only a few such cases where I can legitimately say that "I want this to be its own class but it will only have one implementation that I want to be hidden from consumers".
b) When I have made this assumption in the past it has come back to bite me in the ass about half the time. Granted, this would be addressed by myself getting better at identifying areas of change.
c) If the project is small enough that it doesn't use IoC or even manual DI, TDD altogether is probably overkill.
d) When I have done this (usually a handful of times per project) it's usually been on a low enough level that I don't mind testing both the outer and the enclosed component simultaneously. As a matter of fact, it usually makes more sense to test with that component in place then to mock it.

So where does that leave us?

Example 1 I just don't buy at all. I pride myself on the ability to be convinced of others' points of view but I just don't see it happening on this one.

Example 2 I would agree is valid, but only for a tiny percentage of design decisions. Certainly not a proportion that merits the price-tag and arguably not a proportion that merits the potential of on a late night abusing a Typemock feature to the detriment of my design.

Mocking in legacy applications I agree 100% - Typemock is the way to go and this use in and of itself validates its existence.

As for greenfield TDD apps, I've still to see an example of a pattern that only Typemock handles that I would actually want to use on anything but an edge case. I think static methods to drive a DSL has the potential to win me over but I still haven't been able to think of a concrete example.

Doron said...

Disclaimer: I work for Typemock. I try to post as a professional developer, but my view is obviously colored. You'd be surprised how much thought we put into this sort of things, deliberately being skeptic whether we are balancing power with maintainable design correctly.

@George:
I'm following your reasoning (and I could bring in a few more examples to add to Lior's, but that point is already made), but I don't get the conclusion.

You agree that Typemock is a must for unit testing in a legacy project. The same point can be made for Sharepoint, somewhat for WCF and who knows which other frameworks. Now, you are saying there is a small percentage of cases where Isolator is justified. I respect that, even though I think you are underestimating the number of times you are going to encounter these cases.

Here's where I lost you - you are saying greenfield projects would not need Isolator? You just established it's a more capable tool (slightly in your opinion, more so in mine), and you are going to throw that advantage off the boat? Why not use the best tool at your disposal? I think Lior answered the design considerations pretty well with pragmatism - why still limit yourself in these cases?

George Mauer said...

@Doron
I think I accidentally dragged in a bit of the conversation from Roy's post into here. I should have restated the context.

My understanding of the discussion can be summarized as so:

All things being equal, do the advanced capabilities of Typemock as opposed to the free frameworks tend to lead people to poorer designs?

My response to this is

Tentatively yes. I am as of yet unable to think of a non-edge case pattern that I would want present in my application that I do not use because Rhino Mocks fails to support a scenario for testing it. If this statement is true then I think the temptation of using a feature when I should first look to other solutions could potentially influence my application in a negative manner.

Roy pointed me to this post for examples and I just don't think that the ones here hold much water.

Of course there's lots of cases where having Typemock would be a major boon, no argument there. Most of them have to do with your hand being forced by an external dependency and that is where Typemock shines.

I don't think the argument is at all about Typemock's invalidity as a framework, I am simply asking the question: when would I want to use it on a greenfield application? Put out an info sheet with your list of examples (preferably with DateTime.Now excluded). I will go through it and either decide that the benefits are too few to justify the pricetag or be amused at the previously invisible walls that Rhino.Mocks had erected around me. In the later case I will immediately start bugging my boss for a license.

 
Design by Free WordPress Themes | Bloggerized by Lasantha - Premium Blogger Themes | Walgreens Printable Coupons