A common scenario in any service is you may have some asynchronous operation that you need to perform on a data set of unknown length. Your first instinct may look something like this:

for…of over unknown collection with await in loop

You quickly discover that this is not very fast. Simple interrogation of CPU utilization will show Node is spending a lot of time waiting. Depending on your lint rules, you may not even be able to get this code past a CI build. The solution for many is to simply fire all the promises then await all of them. That, in and of itself, is not a bad thing; for example, say you know that every time you execute a given route, you need to make 3 unrelated calls to other services. Since you know it will always create 3 promises, this is bounded and safe. The problems arise when the promise creation is potentially unbounded.

Promise.all() on an entire unknown collection

You test your code and discover it is much faster, Node is not spending time waiting around, and the lint error is gone. You merge and deploy your code, all the integration tests pass, and you go home to watch The Expanse. Suddenly, right when the Rocinante (a legitimate salvage) is entering a tense battle, you start getting PagerDuty alerts. Your services containers are getting restarted constantly due to Out of Memory exceptions, the container host is experiencing socket exhaustion, and your response times are through the roof. You have a major performance and scalability issue.

Benchmarking unbounded promise scenarios

This is a nice clean scenario, so the suspect code is easily identified, but what if it isn’t? Why does this code fail to scale? This is where Clinic.js shines. For the sake of exercising Clinic.js, I created 6 scenarios that all perform the same asynchronous work some n times using different strategies. They are using for…of, the new ES2018 for await…of, the Promise.all() scenario described above, using Bluebird.map() with a concurrency limit set, the promise-limit module, and the p-limit module.

For the test, I start a clustered web server that accepts a get and post request. Both verbs will perform some arbitrary work before completing the request. In the test itself, I get the data, do arbitrary work, and post the results back. In both cases the work is primarily parsing JSON and manipulating the object in an attempt to simulate a common Node workload. Each test run does this work 500 times, for each element in a mock collection. For the limiting modules, they are all set to allow 10 concurrent promises. All tests were performed using Node v12.14.0. The average execution times over 10 test runs can be seen below.

╔═══════════════╦══════════════════════════════════╗

║ Test ║ Average Execution Time (Seconds) ║

╠═══════════════╬══════════════════════════════════╣

║ await-for-of ║ 6.943 ║

║ bluebird-map ║ 4.550 ║

║ for-of ║ 6.745 ║

║ p-limit ║ 4.523 ║

║ promise-all ║ 4.524 ║

║ promise-limit ║ 4.457 ║

╚═══════════════╩══════════════════════════════════╝

The results here certainly reflect the performance improvement when going from for…of to issuing concurrent promises, but the limiting libraries have similar execution times. Let’s dive deeper into what is going on.

for…of and for await…of

for…of test code