What do you do when an open-source project you rely on no longer meets your needs? When your choice affects not just you, but a larger community, what principles guide your decision?

Submitting patches is often the first option, but you're at the mercy of the maintainer to accept them. If the changes you need are sweeping, substantial alterations, the odds of acceptance are low. Eventually, only a few realistic options remain: find an alternative, fork the project, or write your own replacement. Everyone who depends on open source faces this conundrum at one time or another.

After relying for years on the community-developed mgo Go driver for MongoDB, MongoDB has begun work on a brand-new, internally-developed, open-source Go driver. We know that releasing a company-sponsored alternative to a successful, community-developed project creates tension and uncertainty for users, so we did not make this decision lightly. We carefully considered how our choice would affect current and future Go users of MongoDB.

First, some history: Gustavo Niemeyer first announced the mgo community driver in March, 2011 – around the same time that MongoDB released version 1.8.0 of the database. It currently has over 1,800 stars on GitHub and 32 contributors – including several current and former MongoDB employees. The incredible success of MongoDB in the Go community owes a great deal to Gustavo and mgo.

MongoDB itself is part of this community. As the Go language matured and gained in popularity, MongoDB found many uses for it internally. Some of the projects using it include:

Our remote agents for automated deployment, for backup, and for monitoring.

Our command-line operations tools, like mongodump. (Re-written in Go for the 3.0 server release).

Our home-grown continuous integration system, Evergreen.

Our cloud products, like MongoDB Atlas and Stitch have major components written in Go.

From this experience, our engineers contributed back to mgo: over half a dozen employees have commits in mgo, accounting for over 2000 lines of changes.

But the more we used mgo, the more we discovered limitations.

With our in-house drivers – covering popular languages with deep commercial adoption – we often start driver feature development in parallel with server feature development so that we can test them as soon as the server merges a feature. But as a community project, mgo's feature support generally lags MongoDB server development. More critically, our products that use mgo can't easily test against or take advantage of new server features. Even if we thought that Go didn't yet have critical mass in our user base to justify an in-house driver, our own company's products can't wait for new features.

Sometimes, we patched a private copy of mgo to implement new features we critically needed. This isn't always easy. In 2015, we announced our next generation drivers, built upon a published set of specifications for driver behavior. Because mgo predates this work, its conventions and internals don't match our specifications. When the server implements new features and the driver development team writes specs to match, these new specs assume implementation of prior specs. Developing comparable features in mgo can mean starting from a completely different base.

Not only does mgo have different internal conventions and behaviors than our in-house drivers, it encapsulates these behaviors in ways we found constraining. Usually, encapsulation is a good thing – a sign of good design – but many of our products benefit from low-level access to sockets, wire protocol models and encoding. End-users don't need this access, but we have the knowledge to work with our own communication protocols and message formats safely and to great effect.