Mixins: Separation of Concerns in Test Fixtures

I was working with a tester yesterday to provide access to STAF from within a system test written in C# with MbUnit. Recently we started a project to use STAF to coordinate system tests across our test farm so that we can run tests early, often, reliably and in parallel.

As is common with large stateful systems, there are a limited number of resources that system tests much share among themselves. For instance, our system contains a Hammer box with a limited number of ports that must be shared. Likewise, functional tests often share common test user accounts so those tests must never be permitted to run concurrently or else a pool of suitable test user accounts must be provided. Consequently there must be rules governing which test is permitted to use which resources at which times. Up to now, we have settled those rules statically by hardcoding reservations for certain resources or by trying to time-boxing test schedules to discourage concurrent execution of conflicting tests. As expected, this process is extremely brittle. So moving forward we decided to perform such reservations dynamically using the STAF Resource Pool.

So we started off with some code by Rohit Krishna Kumar to provide a primitive C# wrapper for STAF based on P/Invokes. But we wanted to go further! We wanted the test framework to automatically manage STAF Handle registration and disposal and to support a declarative notation for allocating resources from STAF resource pools.

Today's Problem: How to achieve separation of concerns in a test fixture?

So here's the kind of code we want to write in the test:

[TestFramework]
public class MyTest : STAFTest
{
    [STAFResource("master-machine", "HammerPortResourcePool")]
    public int HammerPort;

    [Test]
    public void MakeACall()
    {
        // do something involving the HammerPort resource we just obtained.
    }
}

STAF integration is really cool, but how can we reconcile it with our in-house Web test framework (similar to Watir but written in C#). Both the STAF integration and the Web test framework integration want to be super-classes of the tests! Both of them need to contribute test fixture setup and framework code to manage their lifecycle so it's going to get tangled.

[TestFixture]
public class TangledSTAFAndWebTest : SimpleWebTest
{
    public STAFMixin STAF;

    public override void TestFixtureSetUp()
    {
        base.TestFixtureSetUp();

        STAF = new STAFMixin(this);
    }

    public override void TestFixtureTearDown()
    {
        if (STAF != null)
            STAF.Dispose();

        base.TestFixtureTearDown();
    }
}

Yuck! So we picked SimpleWebTest to be the superclass and to inject STAF support somewhat ungracefully so we could also use it elsewhere with other superclasses. MbUnit v2 only allows one [TestFixtureSetUp] or [TestFixtureTearDown] attribute each per test fixture so they have to be declared in the base class and made virtual. So in general a few changes had to occur across the class hierarchy to ensure this scenario would work. Awww... if only we had mixins...

[TestFixture]
public class STAFAndWebTest
{
    [Mixin]
    public STAFMixin STAF;

    [Mixin]
    public WebTestFrameworkMixin Web;
}

Well we don't really have mixins, but we can certainly simulate them. And with a little framework support it becomes quite easy.

[TestFixtureMixin]
public class STAFMixin
{
    private object fixture;
    private STAFHandle handle;

    public STAFMixin(object fixture)
    {
        this.fixture = fixture;
    }

    public STAFHandle STAFHandle
    {
        get
        {
            if (handle == null)
                handle = new STAFHandle(GetType().FullName);
            return handle;
        }
    }

    [TestFixtureSetUp]
    public void TestFixtureSetUp()
    {
        Type fixtureType = fixture.GetType();

        foreach (FieldInfo field in fixtureType.GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance))
        {
            foreach (STAFResourceAttribute attrib in field.GetCustomAttributes(typeof(STAFResourceAttribute), true))
            {
                string value = AllocateResource(attrib.Endpoint, attrib.PoolName, true).Value;
                field.SetValue(fixture, Convert.ChangeType(value, field.FieldType));
            }
        }

        // etc...
    }

    [TestFixtureTearDown]
    public void TestFixtureTearDown()
    {
        // similar to above, release resources.

        if (handle != null)
        {
            handle.Dispose();
            handle = null;
        }
    }

    /* Code for submitting STAF requests, allocating resources, etc... */
}

Ooooh shiny! The test framework instantiates our mixins for us and assigns them to fields on the test fixture then incorporates them into the test fixture's lifecycle. In principle, a mixin could participate in test fixture setup/teardown, in test setup/teardown and in resolution of data-bound test parameters (like any fields decorated by STAFResourceAttribute).

There's a lot more though. Why stop at just one level of mixin inclusion? Why not just think of the test fixture mixin as a component that has been resolved, instantiated and installed for us by the test framework acting in the guise of an inversion of control container...

Edit: Made a few cosmetic and grammatical changes to the content.