rqlite is an open-source distributed relational database, which uses SQLite as its storage engine. rqlite is written in Go and uses Raft to achieve consensus across a set of SQLite databases. It gracefully handles leader election, and can tolerate machine failure.

With the v3 release series, rqlite can now replicate SQLite databases on a global scale, with very little effort. Let’s see it in action using the AWS EC2 cloud.

Let’s build our global infrastructure

The EC2 system makes it simple to fire up a global presence, so I did just that. I launched 3 m4.xlarge instances, running Ubuntu 14.04 LTS — one in the USA (Oregon), one in Europe (Dublin), and, finally, one in Australia (Sydney).

I ended up with the following hosts:



AWS Region Public Hostname Public IP Private IP USA (Oregon) ec2-52-36-0-40.us-west-2.compute.amazonaws.com 52.36.0.40 172.31.40.213 Europe (Dublin) None assigned 52.51.50.174 172.31.32.210 Australia (Sydney) ec2-52-64-65-8.ap-southeast-2.compute.amazonaws.com 52.36.0.40 172.31.6.75

Check the network

For the purposes of this experiment, I made the Security Groups wide open, so there would be no problems with network access. Of course, you wouldn’t do this in a production environment.

To get an idea of network latency, I ran some ping tests.

From Oregon to Dublin

From Oregon to Sydney

From Sydney to Dublin Install rqlite

Next, after ssh’ing in to each node, let’s install rqlite v3.1.0 on each node like so:

The source of this script is available here.

Start the leader

Let’s choose to fire up the leader in Oregon, using the EC2 metadata system to retrieve the IP addresses. So I ran the following commands on the EC2 instance in Oregon.

We now have a single rqlite node, with a real SQLite database underneath it. v3.1.0 includes the important options httpadv and raftadv, which allows the node to advertise an IP address other than the address it is bound to — in this case it advertises the public IP address of the EC2 instance, allowing other nodes to contact it from outside the AWS US Oregon region.

This node will accept commands, so let’s create a table and insert a record, using the rqlite CLI.

The rqlite CLI provides a very similar experience to the SQLite CLI. But for full functionality (such as bulk inserts and transaction control), use the HTTP API directly.

Create a cluster

There isn’t much point to a single-node cluster, so let’s add two more nodes. To do that I run the next commands on each of the two remaining EC2 instances:

Global replication!

With these simple steps, we’ve now got a globally replicated relational database. The new table and row have been replicated to the two new nodes. And, of course, every future change made on the leader SQLite database will be replicated synchronously to the other 2 nodes, placed out of harms way, thousands of miles from the leader.

Fault tolerance

This 3-node cluster can tolerate the failure of a node, and if that node is the leader, a new leader will be elected within a couple of seconds. If the cluster was 5 nodes in size, it could tolerate the failure of two nodes.

rqlite exposes a HTTP API, which the CLI uses. This API will also redirect clients to the leader node, if the node contacted is not the leader. You can connect the CLI to any node, and it will transparently redirect to the leader automatically, if necessary.

Practical?

Is this cluster practical? Probably not on a network with such high latency between the nodes. In this set up I managed to insert about 2 rows a second, though if transactions and bulk updates are used, the effective rate is much higher. Running a cluster on a global scale is mostly for demonstration purposes, and to drive development of rqlite forward to the point where it was technically possible to replicate at this scale.

Definitely

However, within, say, a single EC2 Region, the performance will be much greater. And by running an rqlite cluster within the same data-center, with each node on a different rack for reliability, one should see 100s of insertions a second. And all with very easy deployment and operation.

Fun?

Absolutely. rqlite is a serious program, and it shows the remarkable power available to individual developers today. rqlite builds on a world-class database , a proven distributed consensus implementation , and a worldwide public cloud, to provide a globally replicated relational database, with SQLite at its core.

See for yourself