I previously wrote about handling the database server going down with the MongoDB Node.js driver. However, I didn't cover mongoose because there were a couple outstanding pull requests for mongoose that would require changes to the article. In particular, connections in mongoose 4.12 emit the reconnectFailed event. With that in mind, let's dig in to how mongoose reacts when your backend MongoDB topology goes down.

Handling Single Server Outages

For this article, I'll use the mongodb-topology-manager package to stop and start MongoDB. I'll also use the useMongoClient option to make connection options cleaner and more consistent with the MongoDB Node.js driver.

Let's see what happens when the MongoDB server that mongoose is connected to shuts down. The mongoose connection will emit a 'disconnected' event when it loses connection, and 'reconnect' and 'connected' events when connection is reestablished. Mongoose uses the MongoDB driver to manage connections, so it will automatically try to reconnect in the same way the MongoDB driver does. Note that these events only get fired for a single server or mongos, not when you're connected to a replica set (more on that later). Here's a script that uses the topology manager to stop/start a MongoDB instance so you can see these events actually fire:

const { Server } = require ( 'mongodb-topology-manager' ); const mongoose = require ( 'mongoose' ); mongoose.Promise = global.Promise; run().catch(error => console .error(error)); async function run ( ) { const server = new Server( 'mongod' , { dbpath: ` ${__dirname} /data/db/27000` , port: 27000 }); await server.start(); console .log( 'Server started...' ); await mongoose.connect( 'mongodb://localhost:27000/test' , { useMongoClient: true }); mongoose.connection.on( 'disconnected' , () => { console .log( '-> lost connection' ); }); mongoose.connection.on( 'reconnect' , () => { console .log( '-> reconnected' ); }); mongoose.connection.on( 'connected' , () => { console .log( '-> connected' ); }); await server.stop(); console .log( 'stopped server' ); await new Promise (resolve => setTimeout(() => resolve(), 2000 )); await server.start(); console .log( 'restarted server' ); }

Mongoose and the MongoDB driver both do connection buffering, so you need to shut both buffering mechanisms off to make your database operations fail fast when mongoose is not connected. By default, mongoose will wait until you reconnect before actually executing an operation. For example, the below findOne() callback will execute after the 'reconnect' event is emitted.

mongoose.connection.on( 'disconnected' , () => { console .log( '-> lost connection' ); }); mongoose.connection.on( 'reconnect' , () => { console .log( '-> reconnected' ); }); mongoose.connection.on( 'connected' , () => { console .log( '-> connected' ); }); const MyModel = mongoose.model( 'Test' , new mongoose.Schema({})); await server.stop(); console .log( 'stopped server' ); MyModel.findOne({}, function ( error ) { console .log( 'Finished query' , error); }); await new Promise (resolve => setTimeout(() => resolve(), 2000 )); await server.start(); console .log( 'restarted server' );

In order to shut off buffering, you need to shut off both mongoose buffering and MongoDB Node.js driver buffering. We have an issue open on GitHub regarding consolidating buffering options, so please feel free to comment on this issue on the GitHub comment thread. Below is an example of how you can shut off both buffering mechanisms and make your database operations fail fast when mongoose is not connected.

await mongoose.connect( 'mongodb://localhost:27000/test' , { useMongoClient: true , bufferMaxEntries: 0 }); mongoose.connection.on( 'disconnected' , () => { console .log( '-> lost connection' ); }); mongoose.connection.on( 'reconnect' , () => { console .log( '-> reconnected' ); }); mongoose.connection.on( 'connected' , () => { console .log( '-> connected' ); }); const schemaOptions = { bufferCommands: false }; const MyModel = mongoose.model( 'Test' , new mongoose.Schema({}, schemaOptions)); await server.stop(); console .log( 'stopped server' ); MyModel.findOne({}, function ( error ) { console .log( 'Finished query' , error); }); await new Promise (resolve => setTimeout(() => resolve(), 2000 )); await server.start(); console .log( 'restarted server' );

The reconnectFailed event

There are two important options for tuning how long the MongoDB driver tries to reconnect to a single server before giving up: reconnectTries and reconnectInterval . When connected to a single server and the single server goes down, the MongoDB Node.js driver will try to reconnect every reconnectInterval milliseconds reconnectTries times, and give up after it runs out of retry attempts. When the driver gives up reconnecting, it emits a 'reconnectFailed' event. Mongoose 4.12 now properly surfaces the 'reconnectFailed' from the mongoose connection. You can access this event in mongoose 4.11, but you need to reach into the underlying MongoDB driver to get it. Mongoose 4.12 makes it just another connection event.

Keep in mind that when 'reconnectFailed' is emitted, this means that mongoose will never reconnect to your server, even if the server restarts. The 'reconnectFailed' emit is likely a fatal error for your application, it means your database connection will not be re-established without your intervention.

Here's an example of setting reconnectTries and reconnectInterval . The below example executes 2 queries, one while the driver is trying to reconnect and one after the driver gave up trying to reconnect. The query that runs while the driver is trying to reconnect will hang until the driver has emitted 'reconnectFailed', and then give back a 'MongoError: failed to reconnect after 2 attempts with interval 100 ms' error. The query that runs after the driver gave up trying to reconnect will immediately give back a 'MongoError: Topology was destroyed' error.

await mongoose.connect( 'mongodb://localhost:27000/test' , { useMongoClient: true , reconnectTries: 2 , reconnectInterval: 100 }); mongoose.connection.on( 'disconnected' , () => { console .log( '-> lost connection' ); }); mongoose.connection.on( 'reconnect' , () => { console .log( '-> reconnected' ); }); mongoose.connection.on( 'connected' , () => { console .log( '-> connected' ); }); mongoose.connection.on( 'reconnectFailed' , () => { console .log( '-> gave up reconnecting' ); }); const MyModel = mongoose.model( 'Test' , new mongoose.Schema({})); await server.stop(); console .log( 'stopped server' ); MyModel.findOne({}, function ( error ) { console .log( 'Finished query' , error); }); await new Promise (resolve => setTimeout(() => resolve(), 2000 )); await server.start(); console .log( 'restarted server' ); MyModel.findOne({}, function ( error ) { console .log( 'Finished second query' , error); });

Moving On

Mongoose 4.12 has 8 new features, including the 'reconnectFailed' event and single embedded discriminators. Make sure you upgrade to take advantage of the completed discriminator API and improved connection monitoring!