The Mantra:

The mantra of Nexus.js is simple: Utilize everything.

Not a single CPU cycle on any available CPU core is to be wasted. If there are more tasks queued and we’re below the maximum concurrency threshold, then we start a new thread. If there are no tasks available, we exit the thread. If all threads exit, we exit the program.

At the centre of it all lies the scheduler, which makes this happen:

It’s as simple as that!

What this means in practice is that we have a dynamic thread-pooling model, where the system adjusts to the workload it is given within its capacity.

This way, a system with 16 cores will not be using 32 threads all the time, thereby wasting system resources and CPU cycles doing nothing; whilst at the same time, if such a system is hit by 1000 requests per second, all 32 threads will spin up and start aggressively serving requests as soon as they arrive. You can control the number of maximum threads using the ‘concurrency’ command line flag.

It’s even possible to run Nexus with a single thread by passing a ‘concurrency’ value of 0, whereupon it will act like Node, emulating a single-threaded event loop; albeit a little slower, since it won’t perform any asynchronous I/O in the background.

So, what difference does this make? Node can scale using the Cluster API easily.

While that’s true, it won’t be utilizing hyper-threading (or logical processors) to the maximum extent, you will also have to multiply your application’s memory requirements by the number of instances you start.

Let’s say your application consumes 700 megabytes of memory at its peak capacity.

Now multiply that by 4. That’s 2.8 GB of RAM just to utilize four cores. On a small VPS, this means a lot.

Imagine a real-world application that consumes 4GB of RAM to serve live video streams. If you try to scale that using Node’s Cluster API, you’ll have to multiply 4GB by the number of cores.

So let’s say you deploy it on 16 cores. That would be 64GB of RAM, just to utilize half the CPU capacity, with no means of utilizing the other half. (If you take logical processors into account)

Now imagine that same scenario with Nexus. You’ll start a single instance, which will consume… let’s say 6GB of RAM. It will start 32 threads, and start serving requests.

The number of threads will adjust dynamically, and if you’re not serving a high load, some of the threads will exit, thereby saving some system resources. (Not easily done with Node’s Cluster API as far as I know)

The threads will share the memory, and depending on your implementation, you can even reduce memory usage further.

Instead of 16+ open connections to your database server, you’ll have one (it ultimately depends on your pooling solution, but you can now at least control how many connections you want).

Most important of all: Nexus will utilize logical processors to their maximum capacity.