When running MongoDB in production, you may see queries that should be fast, but instead are exceedingly slow. For example, my Node.js apps have seen a findOne() on a collection with only 1 document take over 1 second.

There's a simple explanation for this phenomenon: a MongoDB server can only execute a single operation on a given socket at a time. In other words, the number of concurrent operations your Node.js connection can handle is limited by the poolSize option. For example, the 2nd query below will take approximately 1 second, because poolSize = 1 and it is blocked by a slow query.

const mongodb = require ( 'mongodb' ) run().catch(error => console .log(error)); async function run ( ) { const db = await mongodb.MongoClient. connect( 'mongodb://localhost:27017/test' , { useNewUrlParser: true , poolSize: 1 }). then(client => client.db()); await db.dropDatabase(); await db.collection( 'Test' ).insertOne({ answer: 42 }); const promise = db.collection( 'Test' ).find({ $where: 'sleep(1000) || true' }).toArray(); const startTime = Date .now(); await db.collection( 'Test' ).findOne(); console .log( 'Executed query in' , Date .now() - startTime, 'ms' ); }

You can visualize this by imagining db as a set of poolSize train tracks. If there's only 1 track and a slow, overburdened cargo train takes 1 second to clear the track, the bullet train behind it needs to wait.

In general, there are 5 methods I've used to minimize the impact of slow trains:

1) Increase poolSize

2) Separate connection pools for potentially slow operations

3) Break up one slow operation into many fast operations

4) Indexes

5) The maxTimeMS option

Increase poolSize

Increasing poolSize is a one-liner in both the MongoDB Node.js driver and Mongoose. poolSize = 1 is good for experimenting with the slow train problem, but in production you should use at least the default poolSize = 5 . Below is an example with poolSize = 10 .

const db = await mongodb.MongoClient. connect( 'mongodb://localhost:27017/test' , { useNewUrlParser: true , poolSize: 10 }). then(client => client.db()); await db.dropDatabase(); await db.collection( 'Test' ).insertOne({ answer: 42 }); const promise = db.collection( 'Test' ).find({ $where: 'sleep(1000) || true' }).toArray(); const startTime = Date .now(); await db.collection( 'Test' ).findOne(); console .log( 'Executed query in' , Date .now() - startTime, 'ms' );

However, too many connections can cause performance issues and can cause you to hit OS-level resource limits. So increasing poolSize to 10k is most likely a bad idea. In production, I generally use poolSize = 10 .

Separate Connection Pools for Potentially Slow Operations

Increasing poolSize won't help if you get a large batch of slow queries all at once. But what can help is separate connection pools for operations that are performance sensitive versus operations that can be slow. For example, in the below example there's two connections db1 and db2 . A slow query on db1 doesn't block MongoDB from responding to a fast query on db2 .

const db1 = await mongodb.MongoClient. connect( 'mongodb://localhost:27017/test' , { useNewUrlParser: true , poolSize: 1 }). then(client => client.db()); const db2 = await mongodb.MongoClient. connect( 'mongodb://localhost:27017/test' , { useNewUrlParser: true , poolSize: 1 }). then(client => client.db()); await db1.dropDatabase(); await db1.collection( 'Test' ).insertOne({ answer: 42 }); const promise = db1.collection( 'Test' ).find({ $where: 'sleep(1000) || true' }).toArray(); const startTime = Date .now(); await db2.collection( 'Test' ).findOne(); console .log( 'Executed query in' , Date .now() - startTime, 'ms' );

Break Up One Slow Operation Into Many Fast Operations

MongoDB aggregations count as a single operation. Because the aggregation framework is so expressive, many people end up creating exceedingly complex aggregations that end up causing slow trains in production.

In general, if you have an aggregation that's causing slow trains in production, you should consider replacing the aggregation with Node.js logic that relies on find() . For example, you could replace $lookup with Mongoose populate.

const mongodb = require ( 'mongodb' ) run().catch(error => console .log(error)); async function run ( ) { const db = await mongodb.MongoClient. connect( 'mongodb://localhost:27017/test' , { useNewUrlParser: true , poolSize: 1 }). then(client => client.db()); await db.dropDatabase(); for ( let i = 0 ; i < 10000 ; ++i) { await db.collection( 'Foo' ).insertOne({ _id: i }); } console .log( 'Inserted foo docs' ); for ( let i = 0 ; i < 10000 ; ++i) { await db.collection( 'Bar' ).insertOne({ _id: i, fooId: i }); } console .log( 'Inserted bar docs' ); const promise = db.collection( 'Foo' ).aggregate([ { $lookup: { from : 'Bar' , localField: '_id' , foreignField: 'fooId' , as : 'bars' } } ]).toArray(); const startTime = Date .now(); await db.collection( 'Test' ).findOne(); console .log( 'Executed query in' , Date .now() - startTime, 'ms' ); }

Indexes

MongoDB indexes are a complex subject, but for the purposes of this article the idea is simple: make a slow query fast. For example, you can speed up the $lookup aggregation above using an index on fooId .

const mongodb = require ( 'mongodb' ) run().catch(error => console .log(error)); async function run ( ) { const db = await mongodb.MongoClient. connect( 'mongodb://localhost:27017/test' , { useNewUrlParser: true , poolSize: 1 }). then(client => client.db()); await db.dropDatabase(); for ( let i = 0 ; i < 10000 ; ++i) { await db.collection( 'Foo' ).insertOne({ _id: i }); } console .log( 'Inserted foo docs' ); for ( let i = 0 ; i < 10000 ; ++i) { await db.collection( 'Bar' ).insertOne({ _id: i, fooId: i }); } console .log( 'Inserted bar docs' ); await db.collection( 'Bar' ).createIndex({ fooId: 1 }); const promise = db.collection( 'Foo' ).aggregate([ { $lookup: { from : 'Bar' , localField: '_id' , foreignField: 'fooId' , as : 'bars' } } ]).toArray(); const startTime = Date .now(); await db.collection( 'Test' ).findOne(); console .log( 'Executed query in' , Date .now() - startTime, 'ms' ); }

The maxTimeMS Option

The maxTimeMS option tells the MongoDB server to stop running a query after a certain period of time. In the below example, the slow aggregate() call will throw an error after approximately 10ms. Because the MongoDB server stops executing the slow aggregation after 10ms, it unblocks the fast findOne() .

const db = await mongodb.MongoClient. connect( 'mongodb://localhost:27017/test' , { useNewUrlParser: true , poolSize: 1 }). then(client => client.db()); await db.dropDatabase(); for ( let i = 0 ; i < 10000 ; ++i) { await db.collection( 'Foo' ).insertOne({ _id: i }); } console .log( 'Inserted foo docs' ); for ( let i = 0 ; i < 10000 ; ++i) { await db.collection( 'Bar' ).insertOne({ _id: i, fooId: i }); } console .log( 'Inserted bar docs' ); const promise = db.collection( 'Foo' ). aggregate( [{ $lookup: { from : 'Bar' , localField: '_id' , foreignField: 'fooId' , as : 'bars' } }], { maxTimeMS: 10 } ). toArray(). catch (err => {}); const startTime = Date .now(); await db.collection( 'Test' ).findOne(); console .log( 'Executed query in' , Date .now() - startTime, 'ms' );

One nice detail of maxTimeMS is that it does not count time spent blocked behind a slow train. In other words, if you run a find() with maxTimeMS = 100 , and that find() spends 500ms blocked behind a slow query and then executes in 50ms, you won't see an error, even though your Node.js process waited 550ms for the query. This means maxTimeMS helps you find queries that are actually slow, as opposed to queries that appear slow because they're blocked by other slow queries.

Moving On