Running all tests in solution using MSTest

There aren't many times I truly have something bad to say about MS. Its not that I'm a great fun, but I admit that in most cases they do get the job done (it just takes a couple of tries though).

However, there is one thing that always tick me when working with MS products and that their ability to create product which doesn't work well with anything else besides other MS products. Normally I'm not into conspiracies and would of written it off as simple mistakes, but I'm guessing I just have to much respect for MS developers.

And the thing is that this always happen at the really annoying small things when you least expect it, but when encountered really make you go *#*#*!*!$*!*#!.

and the story goes like this:

At one of my clients I'm helping setup the build server to run all the unit tests the development team is going to write (which is why I was brought in at the first place). After some initial work in configuring CC.Net (which didn't take long at all), I've wanted to add the unit test execution part. so I went on to search for a way to execute all tests in a given solution. Shouldn't be too hard right?

Well WRONG!

After touring the web for some time it appears the Mstest has, how unconventionally, left this somewhat trivial ability out of their tool set (they didn't forgot to put a button in the IDE for doing so). its easy enough to do it if you use TFS as your build server solution, but using anything else means that one need to feed MSTest command line with all the various test dll's explicitly and MANUALLY!

so here are some other options:

1) Taken from Stack Overflow (towards the end) - a way to semi automate this using msbuild based on some naming conventions.

2) MSTest command line can use the vsmdi as their execution input, so one can simply create a list with all existing tests and use that. However now one need to maintain that all knowing list.

3) write up a small utility which based on the data inside the sln file, produce a list of all tests dll's - shouldn't be too hard but who would like to do that?

and the last thing (which i think i will choose at the end )

4) kind of a brute force, but if MSTest can only receive one dll as input, lets give him one. And I'm not actually talking about actually writing all the unit tests inside a single DLL. I'm thinking about using ILMerge to consolidate all DLLS in a given output directory (production and tests) into a giant DLL and feed it into MSTest. (I just hope that it will be able to process that dll)

Really Really ANNOYING!!!!!

Posted in: MSTest,tips,tools