Time Just Got Real at Compose with RethinkDB 1.16

Version 1.16.2 of RethinkDB is now available on Compose. The 1.16 version of RethinkDB, codenamed "Stand By Me", has two important features – real time push for change feeds and a new API for cluster management. You can upgrade now to RethinkDB 1.16.2 from your Compose dashboard.

The cluster management API has some impressive capabilities to allow for reconfiguring, rebalancing on the fly and a range of system tables that let you see how the cluster is configured, statistics and information on performance, table and job details and more. More importantly, it allows us at Compose greater insight into how your RethinkDB is performing and lets us diagnose issues quickly and tune efficiently.

It's the realtime push functionality which is where things get really interesting though. RethinkDB has had change feeds for some time but in 1.16, the developers have moved to a push model for those feeds, rather than polling, and dramatically expanded what those users of those feeds are able to do with them.

To get a feel for what a difference change feeds can make, let's consider the equivalent functionality using MongoDB. An example of it being implemented is Meteor's oplog driver because, to get that real time feed on MongoDB, you have to tap into to replication oplog. Then, you have to handle all the changes from every corner of the database because that's what the oplog contains - every change.

So if you are interested in say, new additions to one collection, you have to write code to work through every database change to see if it refers to that collection, then notify the appropriate bit of your code that that change has happened. It's a lot of work, it needs to be done relatively close to the database cluster and all the routing logic has to be embedded up in that oplog processor. Which is why the Meteor oplog driver is such an impressive bit of work.

Let's compare that to RethinkDB. For a start, changefeeds are built into the client API and server and aren't a side effect of the replication regime. They are designed to be available on a per client basis with the server taking care of any optimization that needs to be done.

How hard is it to configure a changefeed in your application? Well, let's say we have a table called "chat" and we want to be notified every time there's something new or changed in that table. Here's the entire program (for Node.js):

var r = require("rethinkdb"); r.connect({ host: 'localhost', port: 28015, db: 'chat' }, function(err, conn) { r.table("chat").changes().run(conn, function(err, cursor) { cursor.each(function(err,change_doc) { console.log(change_doc.old_val); console.log(change_doc.new_val); }); }); } );

We just use the .changes() method to get a real time update of all the changes and for each change, we get a document with the old and the new values. That was there before 1.16, but now it benefits from a more efficient push mechanism. What has changed with 1.16 is the range of queries that you can tag .changes() onto.

Want to watch a single record for changes? .get(ID).changes() .

. Want to see any new records appearing within a range of id's? .between(LEFT_ID,RIGHT_ID).changes() .

. Want to see only the changes that affect a particular company? .filter({ 'company':'ThatCo Inc'}).changes() .

. Tracking the maximum value of an field? .max('points_scored').changes() .

. How about when the top ten points scorers change? .orderby('points').limit(10).changes() .

It's surprisingly simple and very powerful bringing all the capabilities of real-time change tracking to the client without all the architectural overhead. All that without losing the extensive general purpose database capabilities that RethinkDB already has. If your application is about reacting to changes in your database as its updated, then RethinkDB gives you a new way to process it, in real time - as it changes.

And that's just the start - the developers at RethinkDB say this is just the foundational step as they head toward RethinkDB 2.0. So we're really pleased to be able to get 1.16 on Compose now to let our users explore RethinkDB's vision for the real time web.

In a coming article, we'll build an application using changefeeds in RethinkDB and show you how you can begin to exploit the potential of real time.