From time to time one faces the situation that one of the unit tests written fails inconsistently. The usual scenario is that when run alone the test passes, but when run as part of the entre suite the test blows up.
In a lot of cases this results from some left over from previous tests. Here is an example of such a case (taken from Typemock-Isolator support forums - AAA tests pass individually, fail in suite). The reported error in this case indicates that for a mock setup in a given test was accessed in some other test which kind of confused the given test.
Solving such an issue usually involves a lengthy Search&Destroy in which most of the work involves finding out which of the test is the one which is interfering with the given test.
So here’s some code that should save most of the leg work. I’m using one of the more powerful features of the Isolator framework – Custom Decorator, which allows adding interception points before and after a given method call.
Here’s the code:
public class TraceTestAttribute : DecoratorAttribute
{
//I'm using a dictionary of lists to store for
//each method all the tests in which the code
//was called
static IDictionary<MethodBase,List<String>> _TestNames
= new Dictionary<MethodBase, List<string>>();
public static string GetData()
{
StringBuilder output = new StringBuilder();
foreach(KeyValuePair<MethodBase,List<string>> entry in _TestNames)
{
output.Append(entry.Key.Name + ": ");
foreach (string testName in entry.Value)
{
output.Append(testName + " ");
}
output.Append(Environment.NewLine);
}
return output.ToString();
}
//this is where all the works get done
public override object Execute()
{
List<string> data;
if (!_TestNames.TryGetValue(OriginalMethod,out data) )
{
data = new List<string>();
_TestNames[OriginalMethod] = data;
}
data.Add(GetTestName());
return null;
}
//going over the stack to locate the test name
//we use reflection to find the method decorated
//with some sort of [Test*]
private string GetTestName()
{
StackTrace trace = new StackTrace();
foreach (StackFrame frame in trace.GetFrames())
{
foreach(object attribute in frame.GetMethod().GetCustomAttributes(false))
{
if (attribute.GetType().Name.StartsWith("Test"))
{
return frame.GetMethod().Name;
}
}
}
return "not from test";
}
//this instructs the framework to apply
//the attibute ALL methods (and not just tests)
protected override bool DecorateMethodWhenAttributeIsClass(MethodBase methodBase)
{
return true;
}
}
As you can see this is quite trivial, when applied to a given method at time the method get calls the attribute Execute method is invoked before the call. In that call I’m going over the stack searching for the test method in which we are running and storing it in a big dictionary keyed by the method we are tracking.
At the end this gives me the ability to stop the execution at each point and easily see the table of all the tests that during their execution a specific method was called (directly or indirectly).
(For good measure I’ve added a GetData method which build a string of the entire dictionary)
I hope you find this one useful, leave a comment if you do or if you have related question.
A warning: currently the isolator contains a defect that will cause an exception to be thrown when the method with this attribute is mocked. I’m sure that this problem will be fixed very shortly by the Isolator team.
11/11/2008
Posted in: 
Posts
