Now I’m looking at this mumbling approach instead, and can easily write small specialised clients to look for patterns, and for visualisation with known-latest data, with Cacti spitting out little data announcements as it collects them.

I’m always a little surprised by people who build systems of any kind and then fail to stick in the real time, as it happens, tail the log file and stare at until it makes sense kind of monitoring that’s my first instinct. There are sometimes anguished cries from “analytics” people on one of the web advertising lists whose data analysis vendor isn’t even keeping up with the pace of daily reporting. (daily wtf? if it matters, you have some sense for what it’s doing while the market is open for whatever market you are in)

A single message can be demultiplexed into multiple queues and hence to multiple consumers that can look at it at a slightly different angle. For example, a message published with “server1.disk.nearfull” routing key can be sent to a consumer that will initiate cleaning up old log files on server1, and at the same time can be sent to consumers tasked with monitoring applications running on server1, in case a failover needs to be initiated. All magically happening out of the box, thanks to AMQP.

If you have any code on this and can open source it, I would be very interested in contributing, and in fact I suspect there will be many people who will be interested in something like this.