Exporting Code Metrics from Command Line

A couple of months ago Microsoft released a tool enabling to gather code metrics on code form command line. (detailed post here).

While it took me some time I’ve updated my Msbuild custom tasks project and added an Msbuild task for better integration of the tool into a build cycle.

The task enable the exporting of metrics for a given solution and includes an xsl to transform the resulting xml into an html report.

(older tasks in the project allowed running all tests (using mstest) in a given solution and exporting VS coverage report to an xml)

To download the latest release go to here, and if you have any questions or additional requests contact me either through the CodePlex forums or directly.

Read more »

Posted in: MSTest,tools

Custom Msbuild Tasks - Updates

In the latest Alt.Net session I’ve showed some custom Msbuild tasks I have written to help people who are using the MSTest and visual studio coverage tool.

(The tasks allow running of all tests inside a given solution and the exporting of the Visual studio coverage report into an XML which can be used inside a build server to generate a coverage report)

Today, I’ve uploaded a new version to have released a new version of those tasks. Mainly I just added some XSL’s to generate the coverage report and fixed some minor issues.The new version can be downloaded from here. For full details about the release check this.

Read more »

Posted in: MSTest

Running all Tests in Solution

MsTest is a nice testing framework. However it can be annoying from time to time. a while ago in the past I’ve blogged about the inability of this testing framework to execute all tests in a given sln file.
Recently i had some time and I decided thats its about time that ill do something productive about this (instead of my usual complaining) and I’ve sat down to write a simple msbuild task that runs all tests in a solution.
here is an example of a build script that uses this task:

<UsingTask
 TaskName="RunAllTestsInSolution"
 AssemblyFile="RunAllTestsInSolution.dll" />

<PropertyGroup >
 <MSTestLocation>C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE</MSTestLocation>
</PropertyGroup>

<Target Name="UnitTestsWithCoverage">
 <RunAllTestsInSolution
   MSTest ="$(MSTestLocation)\mstest.exe"
   SolutionFile="Example.sln"
   Parmaters ="/resultsfile:TestResults.trx" />
</Target>

As you can see you just tells it which mstest version to use, the solution file containing the test you want to execute and any additional parameters if needed.

You can download the source code from here.

*UPDATE*
There is a CodePlex project containing an updated source code along with a couple of  other useful Msbuild tasks. Here's the link

Read more »

Posted in: MSTest,tools,Unit Tests

Using CollectionAssert of MSTest

Here is a nice trick to apply when using CollectionAssert.AreEquivalent of MSTest. While the signature of the method is:

(ICollection expected, ICollection actual)

When using them in your code switch between the actual and expected. Pass your execution result as the first argument, and you expected collection as the second. Doing this will give a better more informative error message in during failures.

Lets see an example,

I have a test which ends with this:

var actual = CrapAnalyzer.CreateCrapReport("", "");

var expected = new Dictionary<string, double>();
expected["MyClass.Method1"] = 1.34;
expected["MyClass.Method2"] = 1.22;
expected["MyClass.Method3"] = 5;

CollectionAssert.AreEquivalent
   ( expected as ICollection,
     actual as ICollection );

When run as is I get the following:

CollectionAssert.AreEquivalent failed. The expected collection contains 1 occurrence(s) of <[MyClass.Method3, 5]>. The actual collection contains 0 occurrence(s).

when I switch between the collections i get:

CollectionAssert.AreEquivalent failed. The expected collection contains 1 occurrence(s) of <[MyClass.Method3, 1.12]>. The actual collection contains 0 occurrence(s).

While at first glance both see the same, the first one gives me no added information that is not written in the test code. The second error on the other hand, actually gives a good description on what happened. Reading it I can clearly see that the result value for Method3 was 1.12 instead of 5 expected by the test code. Now I know the actual difference and there’s a good chance that I can understand what went wrong without opening a debugger.

Read more »

Posted in: AUT,MSTest,TDD,tips

Running all tests in solution using MSTest

There aren't many times I truly have something bad to say about MS. Its not that I'm a great fun, but I admit that in most cases they do get the job done (it just takes a couple of tries though).

However, there is one thing that always tick me when working with MS products and that their ability to create product which doesn't work well with anything else besides other MS products. Normally I'm not into conspiracies and would of written it off as simple mistakes, but I'm guessing I just have to much respect for MS developers.

And the thing is that this always happen at the really annoying small things when you least expect it, but when encountered really make you go *#*#*!*!$*!*#!.

and the story goes like this:

At one of my clients I'm helping setup the build server to run all the unit tests the development team is going to write (which is why I was brought in at the first place). After some initial work in configuring CC.Net (which didn't take long at all), I've wanted to add the unit test execution part. so I went on to search for a way to execute all tests in a given solution. Shouldn't be too hard right?

Well WRONG!

After touring the web for some time it appears the Mstest has, how unconventionally, left this somewhat trivial ability out of their tool set (they didn't forgot to put a button in the IDE for doing so). its easy enough to do it if you use TFS as your build server solution, but using anything else means that one need to feed MSTest command line with all the various test dll's explicitly and MANUALLY!

so here are some other options:

1) Taken from Stack Overflow (towards the end) - a way to semi automate this using msbuild based on some naming conventions.

2) MSTest command line can use the vsmdi as their execution input, so one can simply create a list with all existing tests and use that. However now one need to maintain that all knowing list.

3) write up a small utility which based on the data inside the sln file, produce a list of all tests dll's - shouldn't be too hard but who would like to do that?

and the last thing (which i think i will choose at the end )

4) kind of a brute force, but if MSTest can only receive one dll as input, lets give him one. And I'm not actually talking about actually writing all the unit tests inside a single DLL. I'm thinking about using ILMerge to consolidate all DLLS in a given output directory (production and tests) into a giant DLL and feed it into MSTest. (I just hope that it will be able to process that dll)

Really Really ANNOYING!!!!!

Read more »

Posted in: MSTest,tips,tools

Collection Asserts in MSTest (and some more)

Sometime I can miss the obvious.

I've been working with MSTest framework for over a year now and up until today I completely missed the fact that beside the Assert class MSTest framework contains 2 other assert classes :

1. StringAssert - used for string specific asserts like StartsWith, EndsWith and even support regular expression matching
2. CollectionAssert - used for collection operation like Contains, AllItemsAreNotNull and the most useful AreEquivalent.

Naturally NUnit also has equivalent classes (which of course I missed as well).

Now the thing that really ticks me off is that I was asked this specific question twice in the last month and gave the wrong answer. Only today after I allocated 2 minutes to actually google it, I found out about it.

Lesson learnt - For all consultant out there, Don't get overconfident about your knowledge. Even if its in you specific area of expertise. Always assume that you are mistaken and double check your answer (especially if your answer starts with "You cant...")

Read more »

Posted in: AUT,MSTest,NUnit,tips,tools,Unit Test Patterns,Unit Tests