This is a follow up to our recent TV Companion experiment with the BBC Two show, Autumnwatch. Whilst that post concentrated on the experience, this is an overview of the technologies we used for the experiment, and gives you the opportunity to build your own, by providing some simple example code and installation instructions.

First, what is Push vs Pull? In a web context, ordinarily, you would request a webpage using your browser and a some data would be returned, Pull. The only way that you would then subsequently see any changes to that data, would be if you requested it again, or if something on that page requested it for you, maybe via polling. Push is when you create a persistent or long-lasting connection from the browser to the server, wherebye the server can send information to the client, without the client needing to refesh their page. Techniques for doing this include Comet, WebSockets or BOSH.

A common use of Push, is when it's coupled with other technologies to create web based Publish/Subscribe (PubSub) systems–you have one publisher, who can send information to one or many subscribers–and this is exactly what we wanted to do in this experiment. There are many PubSub setups out there, RabbitMQ, Node.js, Redis, Ejabberd, PubSubHubbub to name but a few. For our setup, we decided to use Ejabberd and the Strophe.js JavaScript library.

Ejabberd is a robust and proven XMPP/Jabber server written in Erlang. XMPP (The Extensible Messaging and Presence Protocol) is an open technology for real-time communication. It is more commonly used for instant messenger (IM). If you use Google’s GTalk service, you’re already using XMPP. It seemed to fit nicely with what we wanted to do, as it already comes with a PubSub extension, but it also has many more great features: Presence, Multi-User Chat and is fully extensible (It's XML). Importantly, it comes with a web API that out of the box can understand XMPP via BOSH.

Strophe.js is a JavaScript library for speaking XMPP via BOSH. Put simply, allows you to do XMPP on the web.

Another win for us, was that due to advances in the way modern browsers handle cross domain Ajax, their amazing JavaScript execution speeds, plus the additional CSS3 support, we can use just HTML, CSS and JavaScript and no longer have to worry about cross-domain problems or performance. It did mean that we could only target some modern browsers, but it was an experiment and this made things much simpler, as in previous attempts we have needed to take a different approach.

This setup worked really well and meant that we could set it up quickly and concentrate on the UX, and what we would actually be sending to the clients. XMPP was designed to send small bits of XML very fast, to one or many clients. For our purposes, this meant sending enough information to all the connected clients, so that they could go away and fetch the data required to build the page. This involved some key/value configuration information, and a url to a text file containing HTML, CSS and JS which we could inject into the clients display area to create the slides seen in the previous post. With extra caching, this would prove very efficient. More so, if we had adopted a technique Google highlighted in their post about instant previews, which was posted a bit too late for us to do anything, but very neat.

Because the basic setup we experimented with was so straight forward, and all the software is free and available, I thought it would be useful to show you how to get up and running with your own little PubSub system, so you can experiment with these technologies. Over on the github website I have created a repository that gives you everything you will need. This includes a simple publisher and client, as well as installation instructions for getting you up and running with Ejabberd on the CentOS Linux distribution. If you have experience installing this on other systems, then I’d be happy to add the instructions to the repository, using the normal pull requests.