First MbUnit v3 checkin.

Well, I finally found time to work on MbUnit v3. After tossing around many Object Model drafts, I decided to leave off for a bit and work on some of the easier bits in the framework. So here's the first checkin log message pretty much verbatim.

First MbUnit v3 checkin.
I've been focusing on the runtime framework side of things for a bit because there were a few ideas I wanted to try out there which weren't quite as crazy as the rest so they seemed like a good jumping off point. The test cases are lacking in this checkin. Don't panic!

This checkin exhibits several new design elements on the framework side of things. A common theme is the expansion of MbUnit's range of applicability to include functional system testing. Such tests place a higher burden on the ability of a framework to schedule, monitor, and report on tests / work-units.

1. Framework components are resolved indirectly through services. This makes it possible to mock out, configure or substitute components that would otherwise have been hardcoded in. Moreover, it affords a greater degree of separation of concerns between the framework and the core. The primary use-case for this approach is to facilitate testing but there are certainly other possibilities involving non-default component implementations that might be useful for particular framework extensions or embeddings.

2. The runtime context of the test is exposed to the test framework (and the test) to provide for reflection of the test execution scope, management of scoped test objects, and for interception of scope-related events. The primary use-case is for specialized test framework extensions that perform advanced tracing of tests or that require robust resource reclamation.

3. Multiple concurrent runtime contexts are maintained. Moreover, contexts can be passed among threads (and are usually expected to be inherited by default via the .Net CallContext mechanism). The objective is to provide a firm foundation for two forms of concurrency: a) one test involving multiple threads that must be coordinated and monitoring and that may need to be killed when the test ends, b) multiple tests running in parallel such as for load-test fixtures.

4. Tests have access to greatly expanded reporting facilities. This has come from the observation that many test writers really want to be able to dump out more log information and classify it. However, a more interesting use-case is to support embedding images and other rich content within the test results. That's one of the top features my office's QA team has requested for our little web test framework since they spend hours tracing faults. If this works out, MbUnit v3's test reports have the potential to be much sexier than anything else out there.

5. Assertions are described by Assertion and AssertionResult objects. Each Assertion has a unique id that identifies the kind of test being performed. Moreover it also captures the parameters of the assertion and the user-supplied detail message. This information may be used by a smart GUI application to visualize a failure intelligently such as by displaying actual/expected results side by side.

Assertions are no longer always one-shot deals that throw AssertionExceptions. It is now possible to evaluate a block of code and collect zero or more assertion failures along the way with the _option_ of terminating execution after the first one by throwing an AssertionException (typical behavior). Consequently assertions are now expressed using 3 primitives: Run() runs a Block and collects AssertionResults it generated, Evaluate() evaluates an AssertionCondition for a given Assertion and returns its AssertionResult, and Verify() tallies and examines an AssertionResult and throws an AssertionException on failures unless multiple assertions are allows.

Assert.Multiple runs a Block and groups multiple logically related assertions into a single unit so that more diagnostic information can be gathered about the state of the objects involved. This takes advantage of the ability to capture multiple assertion failures without throwing AssertionExceptions.

Assert.Ignore runs a Block and disables the verification phase of all contained assertions. This is intended as a debugging tool to allow the code to run longer than it might otherwise and so that more diagnostic information can be captured.

Assert.ExpectedException runs a Block and yields a failure unless an exception of the expected type was thrown.

Assert.ExpectedFailure runs a Block and yields a failure unless an assertion failure occurred within the block which effectively inverts the meaning of the contained assertions (well, not quite). This one may be a little gimmicky but is very straightforward. I'm curious to see what people do with it.

Lots more stuff to come!

Today I'm working on the design doc so I don't get too carried away... There's a lot of stuff in my head that I need to dump now that I have some time.