Sunday, 29 November 2009

Unit Testing Singletons

Much have been said about the singleton pattern, a short visit to Google shows that its mainly about why not to use them. However, for some reason, in all projects I have seen up to date I found quite a few usages of this patten. In fact in most places, this pattern is over used extensible. I'm guessing this is mainly due to the ease of implementation, and the fact that in most systems one can find several classes which are only instantiated once. (i.e. we only need a single instance in the system)

This post is NOT about whether singleton are good or bad, for me that's not an interesting question. In realizing that most people use them, my goal in this post is just to show some useful tips on how to actually handle them during tests .

But first lets look at a common implementation of the singleton (taken from Wikipedia):

public sealed class Singleton
{
private static readonly Singleton instance = new Singleton();

static Singleton()
{
}

private Singleton()
{
}

public static Singleton Instance
{
get { return instance; }
}
}
This implementation poses 2 testing problems that need to be addressed:

  1. A private constructor - When writing tests, our main goal is to make them independent. Therefore we prefer that each test case will work on a different instance to avoid any chance one test will affect another.
  2. The static Instance method - its a valid assumption that much of the code will access the singleton using the Instance method. And it being a static method makes it harder to inject a fake instead of the singleton object.

There are several approaches for bypassing these issues:

1) Use reflection - either to create a new instance each test, or maybe to clear the created instance at the end of the test. (a good summary can be found here).

2) Decouple the business logic, from the creation logic - and test the business logic separately. A IOC container is a common technique for doing this, but a simple factory will do just as well.

3) expose some special method for testing, allowing to tweak the internal field - a simple setter sometimes goes a long way.

and I'm sure that there are more ways. to skin this cat.

I, however, prefer to leverage my tools as much as a I can. Specifically lets see how the Isolator (yeah a big surprise) can help me on this.

Scenario one - Faking a Singleton Behavior

You need to test a class which uses the singleton. During the test you need the singleton to behave in a manner which is very hard to simulate using the real production code (of the singleton). you would like to use a mock however its hard to inject the mock into the code since the static Instance method is had to mock using conventional approaches. In this case you can use the Isolator for setting a fake behavior and mocking the static Instance Method. Here is an example:

[TestMethod]
public void FakingSingleton()
{
// Create a fake instance
var fakeSingleton = Isolate.Fake.Instance<Singleton>();
// Set the faked behavior
Isolate.WhenCalled(() => fakeSingleton.SomeMethod()).WillReturn(7);
// Inject the fake into your production code
Isolate.WhenCalled(() => Singleton.Instance).WillReturn(fakeSingleton);
// Execute the test
var actual = Singleton.Instance.SomeMethod();
Assert.AreEqual(7, actual);

}

Scenario two - Testing the singleton class

Using the same technique we would like to test the singleton internal logic. However since the constructor is private its harder to created new instance each test. again Isolator can be used:

[TestMethod]
[Isolated]
public void SomeMethodTest()
{
//Just a cool way to create a regular instance instead of using reflection.
var fakeSingleton = Isolate.Fake.Instance<Singleton>(Members.CallOriginal);

//use this if you want your test to look similar to production code
Isolate.WhenCalled(() => Singleton.Instance).WillReturn(fakeSingleton);

var actual = Singleton.Instance.SomeMethod();
Assert.AreEqual(5,actual);
}

While all this examples are written for the .Net platform, one can achieve basically the same in Java using PowerMock and even (to some extent) in C++ using MockitNow.

Sunday, 15 November 2009

Is Typemock Isolator Evil - Round N+1

Every few months the argument for and against the advance abilities of the Isolator mocking framework burst again. This time it started with this post: Test driven design – Willed vs. Forced Designs

Disclaimer: I used to work for Typemock company and was in charge for a long time over the development of the framework. so I claim no kind of objectivity .

the argument for and against usually circle around the following:

on one side, people claim that Isolator "breaks" the language barriers and, by allowing any kind of design to be implemented will end up in helping to build a poorly designed system.

on the other side, by allowing the freedom to design at will Isolator shift the responsibility back to the developer allowing him to choose the "best" design as he sees fit.

here are some points I want to comment on:

If you need Isolator you have a bad design

Actually that one is in most cases true. However the following claim "you have bad design" also holds for too many cases as well. Yes I'm saying that most systems out there are poorly designed. I'm also saying that most software project out there will fail. What I don't like about the initial claim is the conclusion that usually follows:

If you need isolator when you have poor design, usage of isolator will end up with a poor design.

That statement is plain wrong, you will end up with poor design unless you learn how to design better. The effect Isolator (or any tool for that matter) will have on a team design skills is minimal at best. Pragmatically speaking if you are are working on a legacy system isolator is probably the better choice no matter what. If you are working on a newly project and you just start out TDD, most likely using isolator will increase the chances you'll be able to stick with it. (adding the need to relearn design at this stage is soo much harder), and if you're working on a new project and you do know how to TDD. Then you should just know better and be able to safely use the tool. If you don't then you really do have a problem.

Isolator breaks the natural barriers of the language

Yes it does, but the statement "Statics are the death of testability" which actually means "don't use static methods" adds barriers to the language which is not there. so what's the better approach? again I don't know. It really depends on your personal preferences and has nothing to do with what is the best design

Actually there's no such thing is the best design. Every design must evolve always to fit the system needs.

Show me a concrete example

example number 1:

many systems have a dependency on the time of day. the naive way to approach that is to use DateTime.Now. but oops that one cant be faked (its a static method) making it a real pain to test. so experienced programmer introduce the ITime interface wrapping the system time with an interface and then adding some concrete code to make it work, all for the sake of testability. here's an example taken from the testify project on how to implement this:

public static class SystemClock {

private static DateTime? fixedTime;

public static DateTime Now {
get {
if (fixedTime.HasValue)
return fixedTime.Value;
return DateTime.Now;
}
}

internal static void Set(DateTime value) {
fixedTime = value;
}

internal static void Reset() {
fixedTime = null;
}
}
Please enlighten me as why this is better/simpler as opposed to use of DateTime.Now.


example number 2:

I have a class A depends on Class B. Class B is the ONLY concrete class implementing the interface ISomeInterafce (and most likely this will hold true forever). Class A is the only place (for now and most likely forever) ISomeInterface is used. yes this is a simple example but since I'm inventing it, I get to set the rules.

and there are several ways to approach this:

1. Use some IOC container to instantiate B and send it to A. a valid OO way with many benefits however in my concrete example (and yes I'm the one setting the rules). I don't have a true need for a full pledged IOC yet, and this very simple scenario wont be the one causing me to start using an such a big hammer

2. Use a factory - a simpler version of the IOC container idea, much more light weight. but still at this point of time (and yes I get to make the rules) under the specified circumstance it falls under the YAGNI directive. If the situation changes (i.e. more places will use ISomeInterafce or more "kinds" of ISomeInterafce will evolve) a factory will be used but for now I don't really need that.

3. Instantiate B somewhere and pass it into A (the simplest form of DI) - a very simple solution which is valid under this scenario. the down side for it are , that it exposes some inner working details of A which may not be a good idea AND most likely it only shifts the problem to someplace else.

4. Just use a new statement somewhere inside A (lets say constructor) - I like this way. Its the simplest thing that can possibly work, and I'm not tied to the "design for testability" chains, so I can do that. and just before I get chopped, yes taken outside this context of this simple concrete example is most likely not a good strategy.



I want to make the Choices. I'm the professional trained to do them. I wont stand being TOLD how to do my job, not by a person, and especially not by a tool.

Thursday, 29 October 2009

Gathering worthwhile feedback

Tobias Mayer has got me (and I guess several other) thinking about the common feedback process used by trainers. While I think that there's much to learn from the general purpose form handed out at the end of a training, clearly doing it only at the end of a training session means you cant use it for improving during a lengthy course.

Therefore I use two different techniques for gathering feedback during a course

Course Retrospect

this is actually quite simple and effective technique, towards the end of the second day. usually after I cover the sprint retrospect meeting. I conduct a short "retrospect" meeting with the aim of improving the actual course. My current favorite way is borrowed from Alistair Cockburn and we try to list out (together in an open discussion) the things which are:

  1. Going good for us so far,
  2. causing issues and difficulties,
  3. and things that we want to try to do different on the next day.

That way not only do I gather feedback that I can use to improve on, we also get the chance to practice and witness the power of Sprint retrospect meetings.

Guided Questions

The second technique I'm using was inspired by Roy Osherove, and its a little more subtle. The basic idea is to ask a few (usually 2) open ended questions regarding the passing day. For a concrete example read these 3 posts: day 1, day 2 & day 3.

And this works really well. First, asking these question at the end of the day makes people go over and reflect on what they have learnt during the day. It take 5-10 minutes in which I see them going through the written material trying to fish out an answer. Just for doing this I think its a great practice. Second, I really learn a lot about how they grasp Scrum, what are the main difficulties, what is seen as beneficial, what they think will be easy and so on. And last in most cases the answers show me how well I did as a trainer. For example on the first day I got an answer saying that "Documentation is not needed." That was not the idea I was trying to convey (In fact I clearly specified that "its not that documents are not needed...." at least twice) therefore I need to improve on that (clearing that point again on the following day).

Both technique are simple, quite fast (takes between 5-15 minutes) and results in good feedback. combining them with the general purpose feedback form really works great for me so far.

What have we learnt today (last day)

At the end of the last day of the course I've asked two different questions:

  1. What will be the easiest thing to implement at your company?
  2. What is probably the hardest thing to implement at your company?

Here are the answers I've got (as is):

The easiest thing
  1. Pair Programming
  2. usage of Task Board (General purpose)
  3. Stand up meetings
  4. Creation of product backlog
  5. Agile estimation Techniques
  6. using Pair Programming to help bring a new team member up to speed
The Hardest thing
  1. Automated unit tests/TDD
  2. Division into Sprints (especially short sprints)
  3. Locating people for scrum roles and creating a self contained team
  4. XP
  5. Building a true self contained team.
  6. Pair Programming

Again almost no overlap of the answers (other then a few saying TDD /AUT is hard). I really enjoyed this cycle of the course, the people attending were really great and I would like to thank them for their feedback.

If anyone is interested in attending, next time will be toward the end of December. Just leave a comment.

Tuesday, 27 October 2009

What have we learnt today/ (day 2)

On the second day I’ve added two more questions to the list:

  1. What will be the be the most valuable thing to you?
  2. What is the most useless thing you have heard so far?

Here are the answers I've got (again not edited):

The Most Surprising
  1. A typical programmer manages 4-5 hours of ideal work per day.
  2. The actual Scrum Master role in a Scrum process.
  3. No hard delivery target (just an estimation), we commit
  4. Priority Poker.
  5. using Agile has brought companies to a defect rate of 1 per quarter.
  6. that sprints can be 1 week long including planning.
  7. that people can be involved/do tasks which are not in their main expertise area. (a QA helping a developer to estimate).
The most controversial
  1. the SM doesn’t need to be a team leader.
  2. The option for terminating a sprint.
  3. a team which managed himself for a long period of time.
  4. Vertical planning Vs Horizontal Planning.
  5. a team leader should not be the Scrum Master.
  6. sprint planning should take into consideration expertise of the various team members.
  7. that the team allocate its own tasks.
  8. no defined time for system analysis. no allocated time between sprints for turning the story into a requirement document.
  9. That SM has no authority over the team.
The Most valuable
  1. Techniques for estimations, task allocation within the team.
  2. splitting the work into very short tasks.
  3. Priority Poker
  4. Vertical Planning
  5. Group estimations
  6. not using buffers.
  7. the fixed sprint content that need to be formally approved at the end of the sprint.
The Most Useless
  1. A manager should “keep” a private buffer and should be completely transparent with his team.
  2. not allow people to multitask.
  3. too much QA in the process (i don’t have enough QA people)
  4. The Claim that “The Plan is worthless, Planning is important”
  5. Team leader as the SM of a different team.
  6. picking SM’s which are not team leaders.
  7. that the team leader is not involved in the process of estimations which should be left to the team only.

What amazes me time and again that there is almost no overlapping between the different answers.

Is Scrum good for me?

The second question i get form most people (after "Does it actually work?") is does the Scrum process good for me?

Mike Cohn has showed a great technique for evaluation which of the processes out there is most suitable for your project. you can find the details here.

Sunday, 25 October 2009

What have we learnt today?

I've started today another round of the Practical Scrum Course. At the end of the first day I search for some feedback in the form of two questions I ask:

1) What was the most surprising thing you have heard today?

2) What is the most controversial thing you have heard today?

here are the answers I've got (not edited):

The Most Surprising

  1. No true need for comprehensive documentation.
  2. Estimations are done in units which does not represent time.
  3. "Done" should be defined, sounds trivial but when one think of it ...
  4. Agile can works for large groups.
  5. Waterfall was presented as an example for a flawed process.
  6. Scrum can be an appropriate process for most projects (90%)
  7. Agile and Scrum, are established and well defined methodologies.
  8. No Clear definition of who is the PO

The most controversial

  1. Scrum can work for "Real Time" projects.
  2. One of the goal of an Agile Process is to have Fun.
  3. Velocity can't be compared between different teams.
  4. Short iterations results in intensive management, in which we repeat many steps which lead to big overheads. (this is in response to the idea that short iterations are the bets way to develop software)
  5. Stories should/can be represented by one sentence in the "customer language".
  6. a process can work without "Requirements Documents"
  7. Documentation is not needed.

Clearly there are some things I need to go over and refine again. However I also think that we are making progress.