Gallio Build Server Down for Maintenance

The build server has been in an unhappy state for quite some time now (registry corruption due to frequent crashes of the VirtualBox VM).  Now it finally seems to have gone over the edge.

As a result I’ll be rebuilding the build server sometime in the next week.  I think I'll set the new one up a bit differently if I have time.

Long Term Ideas

1. Split up the build based on component dependencies.

Gallio ships with plugins for ReSharper 3.1, 4.0, 4.1, 4.5 and 5.0.  The trick here is that in order to build and test those plugins, all 5 versions of ReSharper need to be installed.  Ouch!  (Ok, we will probably be dropping 3.1, 4.0 and 4.1 soon…)

Of course Gallio also ships with plugins for NCover, TypeMock, AutoCAD, TestDriven.Net, Visual Studio 2008, Visual Studio 2010, etc…

Imagine all of those components installed on the same VM?  Yeah… it’s a little scary.  Fortunately most of them do not conflict with the others so it mostly works out fine.

Ideally we should just build a bunch of VMs with different groups of dependencies and build them all independently.  That way we can add new dependencies at will without worrying quite so much about how they interact (or worry about the 12 hours it will take to reinstall everything if the VM blows up).

2. Multiple Platforms

We should really be testing on at least the following platforms:

• Windows 7 64bit
• Windows 7 32bit
• Mono on Windows
• Mono on Linux

Basically that means we’ll need a whole bunch of VMs for various configurations.  Tricky.

3. Add White Box Tests.

Right now there are a bunch of things we don’t test rigorously in an automated way.  Here are a few of them:

• The installer.
• The Visual Studio templates.
• The Visual Studio add-in, besides checking MSTest integration.
• The ReSharper plugin, besides integration tests for the reflection API.
• The UIs, besides unit tests for the Models and Presenters.

I spend a lot of time before each release manually testing this stuff.  It just doesn’t scale.

All we really need are a few tests to get sufficient confidence that the system isn’t broken in really boneheaded ways.  For example, Visual Studio shouldn’t pop up a dialog complaining about the Gallio add-in encountering an error during initialization.  (Usually that means the installer is broken somehow.)

4. Faster Release Cycles

All it should take for a release is for us to spend a couple of minutes writing release notes, adding some more documentation to the wiki, and pushing a big red button to promote the current nightly build to stable.

Paradoxically, the first step in this process is that unstable builds need to be more visible.

What I mean is that we should maintain consistently up to date release notes and documentation for our pre-release builds too.  We should get more people using those pre-release builds and perform more sanity-checking with automation so we’re not chasing last-minute bugs.

When it comes time to release, all we should have to do is post a new stable build label to the website and spam the newsgroups.  :-)

Duh.

Technorati Tags: mbunit,gallio