Friday, March 28, 2008

Do we need [TestFixture] anymore?

The [TestFixture] attribute has been a fixture (pun intended) of unit testing in .Net since NUnit was released.  It simply designates that a class contains tests and possibly some code to set up the test environment before tests and tear it down after tests.

Various alternatives have been proposed:

  • MbUnit v2 introduced a variety of different kinds of fixtures, such as [TypeFixture], to capture different test lifecycles.
  • MSTest calls this thing a [TestClass], reflecting an emphasis on syntax.
  • NSpec replaced [TestFixture] with [Context], reflecting a change in testing philosophy and outlook.
  • xUnit.Net eliminated [TestFixture] altogether, since it is sufficient to know that a class contains tests to infer that the class is a fixture.

Now with MbUnit v3, we have unified data-driven testing concerns.  The perennial [RowTest] attribute has been eliminated because [Test] is itself adequate for describing a data-driven test.  (You can simply replace all occurrences of [RowTest] with [Test] and it will work just as it used to in MbUnit v2.)

Likewise we don't need [TypeFixture] because an ordinary [TestFixture] now supports data-driven features that are a superset of what [TypeFixture] and its fellows once provided.

Should MbUnit keep [TestFixture] at all?

I think it should still retain some attribute to denote that a class is a fixture.  Despite the incredible power of the humble [TestFixture] today, there can still be special kinds of fixtures.  So I prefer keeping things explicit.

However, there's one more detail.  In MbUnit v3 (well, really in Gallio) there is no concept of a test fixture.  Tests are recursively composable.  A fixture just happens to be a test that contains other test.  In fact, fixtures can contain nested fixtures which are subject to the same setup/teardown rules as any other tests they may contain.

Test fixtures are not primitive.

Confusion

We could replace [TestFixture] with [Test] and all would be well.

My concern is that it might not mesh very well with the mental model of testers.  There are also quite valid historical reasons to avoid changing the syntax too much.

What do you think?

Tuesday, March 25, 2008

Great recruiting ideas

With Gallio Alpha 2 released, I'm about to jump into ultra-recruiting mode at Ingenio.  I've got two spots to fill, one for a QA Automation Engineer and one for a Platform Tools Engineer.  More on that in another post later.  (Interested in working with me on MbUnit and other fun stuff?  Let me know!)

Meanwhile I just came home from a lovely day trip to Sausalito with my lovely wife to find two things:

  • 1. People seem to like Gallio.
  • 2. An Apple recruiter sent me a nice and friendly email and voicemail regarding a potential position on the iPhone team.

That's right.  Download the iPhone SDK, get a recruiter call!  I have to give credit to Apple for this great idea.  (And here I thought product registration information was mostly used for sales and demographics.)

Hey, I wonder if they'd let me work on porting .Net to the iPhone?  Hmmm...  Maybe bridge with Objective-C or Java and have a go at leveraging Gallio in this space for testing.  Alright, that's perhaps a little far-fetched but I've seen stranger things.

This puts me in an interesting spot though.  I can keep my great job at Ingenio leading a team and working on all sorts of platform infrastructure and Open-Source derivatives thereof, I can keep exploring the iPhone SDK with possible start-up ventures in mind, or I can develop the iPhone SDK and hopefully rub shoulders with some of the great designers at Apple.

Or I can try to do it all at once... Muah hah hah hah!

Edit: It turns out these events may have been coincidental.  Still, this does give me some ideas for recruiting in the future...

Monday, March 24, 2008

Announcing Gallio v3.0 Alpha 2

Today we have released Gallio v3.0 Alpha 2, including a preview of MbUnit v3.

Go visit our new website for the download: www.gallio.org.

New and Notable

1. MbUnit v3 Data-Driven Tests

There's been a ton of work behind the scenes to provide a compelling model for data-driven testing.  Here's an example of this feature in action.  Notice the use of generic method parameters.  Also notice that we perform an automatic string to DateTime conversion and that we take some care in how we format the parameters for presentation in the test results.

A generic test for collections

The results

2. ReSharper Integration

Run tests inside ReSharper.  The Gallio plugin for ReSharper currently supports MbUnit v2, v3, and xUnit.Net.  It will of course support any test framework with a suitable Gallio adapter.  For example we have a prototype of a native NBehave implementation on Gallio with full ReSharper integration that just works.

An MbUnit unit test session in ReSharper!

3. Icarus Test Runner

Development on the GUI is progressing smoothly.  It is not quite feature complete but it is coming along quite nicely.

Icarus

4. Integrated support for TypeMock and NCover

This is all it takes to run your tests with NCover or TypeMock using MSBuild:

<Gallio Assemblies="MyAssembly.dll" RunnerType="NCover" />
<Gallio Assemblies="MyAssembly.dll" RunnerType="TypeMock" />

The trick is that the tests will run in an isolated host process with the profiler attached.  You can of course plug in your own runners too.  You can of course pass the same arguments to the NAnt task, PowerShell task and the command-line runner (Echo).

Magic.

5. Visual Studio Templates and ASP.Net MVC Support

We've made it easier to get new projects off the ground by providing Visual Studio project and items templates.  What's more, we also support integration with the new ASP.NET MVC unit test project Wizard!

Creating a new test project

The ASP.NET MVC new project wizard with MbUnit v3 shown

6. New Website

Mark did some great work on www.gallio.org.  Please tell us what you think.  We'll be busy over the next few weeks filling it up with content, including documentation.

And lots more...

Really.  Lots.  We've got a book project underway and tons of little details just below the surface like automatic data conversion, rich formatting, MHTML reports, isolated test host processes, embedded images in CCNet reports, new API, a Pex package and more.  Expect more blog articles coming your way.

So try it out!

Caveat: This release is an Alpha.  It is a high-quality Alpha, but an Alpha nonetheless.  It is not feature complete but it is quite featureful!  I am indeed using it in production.  But your mileage may vary so please send us your feedback!

Technorati Tags: ,