Using Message Queues Will Improve Your Life

When you’re first frantically trying to get version 1.0 of your project out of the door, it’s easy to avoid reading about concepts that are new to you: “Naaah… it’ll be quicker for me to do it my way! is an all-too-common thought that often runs through my head.

By way of example, this is how I felt whenever I was coding a feature that involved executing a task in the future, such as scheduling a status update to be sent to Twitter. Simple enough: create a table in the database for generic tasks, and a cron job to run them at regular intervals.

And then I read this fantastic blog post about why doing the above with a database is not a good again. As has happened many times, “You’re a fool” began running around in my head instead.

For my latest project, Interactwive, I wanted to see if I could make life better. I spent 20 minutes looking for a system that’d keep my life simple: the minimum of effort on my current setup, and must support PHP. I found that activemq works on Windows, so that took care of my development setup, and it implements the Stomp protocol, which makes message queuing devilishly easy with PHP.

I expected a bit of pain… but there wasn’t any. This made me suspicious, and speaking honestly, several weeks later I’m still expecting it to blow up any second now. Technology that does what it says and works straight away? Witchcraft!

Just by way of example: let’s say you have one PHP script that processes tasks. Things start to get busy, and so you want two or three scripts to processes tasks. With activemq and Stomp, this was as easy as firing up the three scripts… and that’s it. Far too easy!

Now starting to see the light, I had a spare couple of hours this weekend, so sat down with TweetingMachine, determined to switch it across to using message queues. And you know what? It took about an hour.

In total, I’ve spent about two hours learning how to use activemq and Stomp. In return I now have a vastly happier server, far more maintainable code, and lots of other nice extras, such as error notification and logging being a doddle.

It’s always irritating to look back and realise how much time one has invested in less-than-ideal solutions. I could have saved hours of frustration – and worrying about scalability – if someone had told me how easy this would be instead.

Move to message queues. You won’t regret it.