An MSTest API Rant.

I just happened to look at the MSTest API today trying to get a feel for whether there were any nicer points of integration other than wrapping MSTest.exe.

Internals All The Way Up

Along the way I happened to notice that the Microsoft.VisualStudio.QualityTools.Common assembly has 166 InternalsVisibleTo attributes on it!

All of the useful API for loading and running tests is internal.  It's possible to create new test types but it does not seem possible to manipulate the test execution pipeline at all.  The good parts of the internal Tmi class are locked away and inaccessible through the public ITmi interface.  I don't see any other API that provides access even indirectly to the LoadTests methods except for the console application.  Disappointing.

Alright Microsoft, I get the message: Keep out.  Fine!

Avoid MSTest

If you are currently considering adopting MSTest internally, I strongly recommend you choose more open/extensible tools instead.  Microsoft is deliberately curtailing the ability of third-party tools integrators to extend their offerings in all but the most trivial ways.  This will ultimately limit the kind of support you will be able to receive long-term from the developer community.

Vendor lock-in is not good for anyone.

It just works.

I've been very busy working on Gallio for the past month.  Sorry for the lack of updates.

Today I had an "it just works" moment which gave me great joy and pride in my work.

I was developing a suite of tests for a bunch of components that all implement the same interface.  So naturally I used the Abstract Test Pattern.

When I was done, I right clicked inside the abstract test class and picked Run Tests using TestDriven.Net.  And lo, it ran all of the tests from all of the subclasses!

I wasn't actually expecting it to do this.  The MbUnit v2 addin for TestDriven.Net did not support this behavior and would throw an error because it did not know what to do with the abstract class.

The Gallio addin supports abstract classes just fine as a consequence of its test filtering strategy which supports matching test types with by inheritance.  Somehow despite knowing it was designed this way, I had reverted to my old inefficient MbUnit v2 habit of picking specified concrete test classes only.

Silly me.  I built this tool.  It's pretty smart...