Monday, March 16, 2009

Gallio v3.1 may use a message bus.

After hanging out at QCon San Francisco last fall with the likes of Dave Laribee, Greg Young, Dru Sellers and Chris Patterson, I got to thinking about some problems I've been having with Gallio's communication patterns.

Why Use a Message Bus?

1. One-way asynchronous message passing is useful for publishing incremental status updates, notifications and interruption requests during the execution of long-running processes.   Doing this with remote procedure calls requires callback interfaces, bi-directional channels, and possibly a background notification queue, ugh.

2. Message passing is explicit about connectivity.  This is a good thing because communication faults should be handled differently from other application-level faults.

3. Assuming the messaging protocol is non-proprietary then we can open up participation to non-.Net agents.

4. Multiple subscribers can listen for messages published to a common message bus.  Multiple publishers can publish to the same message bus.  More importantly: they don't need central coordination.

5. A subscriber's lifetime is decoupled from that of the publisher and vice-versa.  New subscribers and publishers can join the bus at any time which opens up all kinds of possibilities for auditing, reporting, parallel execution, passive monitoring, and maintaining a test grid.

6. Cross-Process communication becomes completely straightforward.

7. The API can evolve dramatically in very useful ways.  In particular, testing can become an open-ended activity rather than a single long-running process.  If you like, you could have testing processes running on a grid all day publishing results from their latest exhaustive trials.

Which Message Bus?

I don't know.

Help me choose.

Here are a few criteria:

1. Should run cross-platform, at least under CLR and Mono.

2. Should not depend on proprietary / non-free infrastructure components.

3. Should not be tightly coupled to .Net protocols such as object serialization.

4. Should be possible (and easy) to implement clients in different languages and runtime environments.

5. Should use well-known and straightforward standard message formats like XML or JSON with as little extra noise in the envelope as possible.

6. Should not require any up-front installation or configuration.  Each endpoint can be self-managed if desired.  The whole bus might be considered transient, something to be set up and torn down as needed.  In fact, we probably don't need a full-blown bus.  We just need some kind of efficient inter-process asynchronous messaging fabric.  (peer to peer?)

7. Should support auto-discovery of services among a group of related processes.

8. Should be able to inform a client of communication faults but does not required durable messaging.

Technorati Tags:


Neil Robbins said...

You might want to take a look at an AMQP messaging system like RabbitMQ.

Jeff Brown said...

Looks interesting. I'll check it out.

rds said...

Also Apache ActiveMQ

Randall Sutton said...

Also check out ZeroMQ