Efficient pub/sub with µWebSockets.js

A scientific comparison using stock exchanges and Socket.IO

Pub/sub is a well known and popular abstraction for working with real-time messaging. You subscribe to a set of “topics” and receive messages published under those. This simple abstraction makes it easier to build many real world applications without having to manage socket groups manually.

Sadly, most if not all popular pub/sub implementations in the Node.js ecosystem are exceedingly inefficient as compared to what is technically possible, and can greatly reduce server performance as load increases.

The most well-known example has to be Socket.IO. It markets itself with the bold line “FEATURING THE FASTEST AND MOST RELIABLE REAL-TIME ENGINE”, managing to cram in two superlatives in one single sentence.

Of course this is a complete lie, not rooted in any scientific evidence or tests of any kind. It’s just made up hyperbole which few web developers care to investigate.

To show this we’re going to create a simplified stock exchange in Node.js, using pub/sub from the newly released µWebSockets.js v16.0.0. Buying and selling equity at fast pace requires optimized server software, and is thus a good real-world example for measuring performance.

Trading bots will buy and sell equity at fast pace and we will then compare performance stats with an identical solution written using Socket.IO.

Like in any televised cooking show, I already made the sources available here. Have a look especially here to see the use of uWebSockets.js pub/sub functions. Respective Socket.IO code is here.

A set of shares are held by the server. Clients connect and subscribe to the valuation changes of a randomly selected share, the share’s topic. Clients are either passive watchers or active traders. Active traders buy/sell shares every 1 ms while passive traders only watch. In the above case 450 passive watchers and 50 active traders compete trading shares. When a share is bought or sold it either increases or decreases 0.1% in value. This value change is published under the respective topic. Performance is measured in transactions per second and reported by the server every second:

If you’re decent at math you may think 50 trades every 1 ms should equal 50k trades per second, and yes you would be right. However for this basic benchmark I used the “websockets/ws” client, which is not fast enough to keep up with that pace. Reported performance is thus not entirely accurate for uWebSockets.js, as the server only runs using very little CPU-time. With increasing client count the difference between Socket.IO and uWebSockets.js becomes too big to easily convey, hence the limited client count.

But why?

There are many technical reasons for Socket.IO, and most similar projects, being so slow. First off 1-to-1 messaging has to be fast. If you’re going to send tons of messages to tons of clients a la pub/sub you need fast message I/O. In the case of Socket.IO this couldn’t be further from reality, as it is composed of several layers of inefficient wrappers.

Socket.IO is a wrapper of Engine.IO which is a wrapper of “websockets/ws” which is a wrapper of the net module which is a wrapper of libuv which is a wrapper of system calls. Every layer adds overhead, particularly at the dynamic JavaScript level.

But it doesn’t stop there, fast pub/sub requires clever algorithms and not just fast I/O. Depending on algorithmic decisions pub/sub can end up snowballing in time complexity with increasing socket counts, or behave close to linear in time complexity.

Most pub/sub implementations scale like a snowball; the more clients the worse it continues to scale and perform. You can think of this as the “bubble sort” of pub/sub; the naive implementation without any prior planning. It’s just a for-loop with send syscalls, potentially iterating over the same sockets many times over.

Think of a mailman having 100 letters to deliver to 10 houses. The naive solution is similar to going round all 10 houses, leaving 1 letter every round, going back only to repeat until done.