Node.js, in a nutshell, is a single-threaded environment for JavaScript execution on the server side. JavaScript you write is executed on one single thread. How then Node.js handles load and keeps performance up? The key is non-blocking asynchronous I/O. To be honest, it is not that simple at all. There is also libuv’s thread pool, which makes certain blocking things act like non-blocking. This way or another, in Node.js we do things asynchronously, which let thousands of clients to be served simultaneously. There are a lot of decent articles describing these aspects in detail, thus I am not going deeper with clarifications.

Eventually, you deal with Promises, which represent some asynchronous action. As callbacks can easily be converted into Promises, I am not going to consider them. I’d like to focus on a particular thing: handling asynchronous operations in parallel in Node.js and handling it right.

Since Node.js 4 (initially with --harmony flag), we’ve been granted native Promises. MDN describes Promise as follows:

The Promise object represents the eventual completion (or failure) of an asynchronous operation, and its resulting value.

One thing should be kept in mind: once a Promise is created, its execution is already happening, no matter if you .then it or not. Once you call fetch('https://example.com/') , the request will be fired; .then or .catch is needed in order to handle the response only. This way, if several Promises are created (for instance, we’ve called fetch synchronously several times one after another), requests will be fired in parallel.

Let’s introduce a helper function and some utilities for demonstration purposes:

numberOfOperations : amount of asynchronous operations to be executed.

: amount of asynchronous operations to be executed. listOfArguments : contains arguments passed to asyncOperation per execution. Introduced for simplicity.

: contains arguments passed to per execution. Introduced for simplicity. asyncOperation : a function that fakes an asynchronous operation: returns a Promise, which will be resolved in 1 to 10 seconds.

: a function that fakes an asynchronous operation: returns a Promise, which will be resolved in 1 to 10 seconds. listOfDelays : array containing amount of seconds to resolve for each asyncOperation correspondingly. listOfDelays is filled once at start and is used for all the takes and subtakes further. Please keep in mind that all the time measurements given below are bound to a specific listOfDelays and may vary between executions. Anyway, the eventual result will be the same on average.

: array containing amount of seconds to resolve for each correspondingly. is filled once at start and is used for all the takes and subtakes further. Please keep in mind that all the time measurements given below are bound to a specific and may vary between executions. Anyway, the eventual result will be the same on average. watchCounter : a function to watch amount of promises executing in the moment. Writes its data to console each second.

Parallel execution — take 0

Let’s take a look at the following code. We do asyncOperation and return list of results.

To be honest, it is not parallel at all. We sequentially wait for each asyncOperation to complete, thus list of results is given in the correct order. We create a Promise on each iteration and wait for its completion. At any given time we have only 1 Promise executing:

1 Promise is executing in every single moment of time

This type of execution took 142 seconds to complete. Execution time is a sum of all the operations execution times. This approach, even though we don’t block, doesn’t seem to be a good solution.

Parallel execution — take 1

It’ll be parallel this time, I swear. Let’s change the previous example a bit. Instead of calling and awaiting asyncOperation on each iteration, let’s call all of them beforehand. It will give us a list of Promises ready to be awaited:

Now we have all Promises started and executing in parallel. We harvest the data awaiting each Promise, which gives the correct results order in the end.

Amount of executing Promises decreases with time:

Starting all the Promises in parallel and harvesting in the end

This type of execution took 9 seconds only. Much better! Now the overall time is bound to execution time of the slowest operation.

Parallel execution — take 2

We can shorten the previous code with Promise.all . It accepts an array of Promises and resolves once all the Promises are resolved or rejects if any of Promises is rejected.

The execution time and chart are almost the same so I am going to skip it. Check the previous take.

Although we are doing good, there is a threat we have to keep in mind.

Parallel execution — take 3

Previous two examples do the thing we want: handle parallel Promises execution. But it executes all the Promises you’ve supplied in parallel, which may cause certain amount of issues.

What if asyncOperation requests third party API, which has rate limiting? It can be whether amount of requests or amount of data passed. You may bump into this depending on amount of Promises.

The point is: doing things in parallel is good but you should consider the actual operation and prevent possible negative consequences of doing it concurrently. Would be great to have an ability to restrict amount of requests executed in parallel. It would allow to control the flow of application much better.

Subtake — 0

What we can easily do is split amount of invocations in the very beginning. Let’s set a threshold to at most 5 Promises executing in parallel at any given moment of time. This way, let’s spread arguments list to list of batches by 5. Next, we call fn for each argument to get a list of Promises. And, finally, we await this list to complete before getting the next one.

Well, can’t disagree, we’ve accomplished what was needed. The amount of Promises executing in parallel is restricted to be less than or equal to 5 at any given moment. We process data batch-by-batch and get the result. The overall execution took 40 seconds, which is worse than 9 but is much better than 142. However, let’s take a look at the chart.

Spread arguments by batches in the beginning and await for each batch to complete

There are peaks of amount of Promises executing. It means the load is not spread and we probably have some resources in idle between peaks. The issue is that a single batch processing takes as long to complete as the slowest operation in the batch takes. Batch may contain 5 operations, 4 of which take 1 second to complete and the last one takes 10 seconds. As we are using Promise.all , the whole batch will also take 10 seconds to complete.

Subtake — 1

In order to correct this, another approach should be taken. Instead of spreading initial array of arguments to batches, let’s process the data Promise-by-Promise. In the previous subtake the logic was:

spread arguments to list of batches and for every batch sequentially do the next steps;

call fn for each argument in a single batch to get a list of Promises;

for each argument in a single batch to get a list of Promises; await a list of Promises to complete;

Now we will change the logic. We will create an initial batch of resolved Promises and chain the following action to each of Promises in batch:

get next argument from arguments list;

call fn ;

; chain the same steps to the returned Promise (if we have arguments to process).

This way, once one of the currently executing Promises is resolved, we take the next argument, call fn and chain the Promise. And as initial batch has the size equal to our concurrency limit, we won’t have more than 5 Promises running in a single moment.

Thereby, we again have at most 5 Promises executing at any given time but now we’ve spread the load better, no peaks present:

Restrict amount of Promises executing to 5 correctly

The overall execution took 33 seconds to complete. We won 7 seconds and spread load much better.

Nice, we now have the execution paralleled and finally gained control over the process. Unfortunately, it is too early for champagne. Hope you’ve noticed we are missing one important thing: no results were gathered upon execution! Let’s fix this. It can be done in several different ways, I’d like to use the fact that we know exactly how many elements should be in resulting array. So I am going to allocate an empty array and fill it with values in correspondence to initially passed arguments eventually.

Each Promise add an entry to a specific place upon completion, thus the order corresponds the initial arguments passed.

So far we’ve accomplished what was needed: the asynchronous operations are executed in parallel and we are able to control amount of those operations running. The code may be shorten in plenty of places but left as is in educational purposes. In addition to that, there are a lot of other things to improve:

asyncOperation is called with a single argument only and we would probably want to pass several in future;

is called with a single argument only and we would probably want to pass several in future; there is no error handling at all, which obviously is mandatory to call it a production-ready solution;

another approach may be taken in take3subtake1part0 and take3subtake1part1 : instead of enhancing and copying array of arguments and exhausting it, we could work with indexes;

and : instead of enhancing and copying array of arguments and exhausting it, we could work with indexes; we could omit Promise.resolve usage and start with real operations instead. Tip: we would need to watch initial amount of arguments;

If someone found this article interesting, I’d suggest applying this enhancements by himself just to warm up a bit.