From Mongoose Models to a Real-time API with FeathersJS

The story of a migration

A couple of months ago, I started working on a collaborative platform for people to efficiently work together on projects, with some cool integrations like Slack to notify users about important events from the Slack team, and Trello to keep an eye on the project realisation.

To make things simple, we’ll focus on the questions & answers module, a simple feature similar to StackOverflow where people can ask questions and get answers.

As a user, I can see the list of questions,

ask a new question,

add an answer to an existing question,

I am notified about a new question or a new answer

An internal API exposes a resource Questions and uses 2 existing mongoose models.

Question model before the migration

You'll notice there is no reference to answers in the Question model.

The relationship is on the Answer model side.

Answer model before the migration

Here comes the controller. Basic CRUD methods for the questions and 2 extra methods for the answers (collection).

Question controller before the migration

Let's focus on the last method addAnswer. When implementing it, there was a confusion between the concerns of controller, service and model. Here is what happens within this controller method:

request parameters are checked (i.e question id)

mongoose documents are created / updated

some emails are sent

the (request) response is sent

Whoa, that’s a lot of stuff happening in one controller!

A 30-seconds polling system was used for the notification system. Even if the realization wasn't the best (the mongo database handled lots of queries, with population), it worked well … until a new feature was requested:

"Now, the user should be able to see a new question or a new answer in real-time"

It was a good opportunity to update the models to match the data expected for the user notifications.

Choosing a solution to make the API real-time

KISS first, DRY second.

APIs have existed for a long time, the REST conventions made them “unified” (kind of a lengthy topic so I won’t go into detail). So for a given resource, we all know the routes that should be created, the actions requested and the kind of responses we should expect. Instead of manually creating them for exposing our resources, I wanted to find and trust a module that does it, and does it well. Something had to be out there, with a real-time feature.

I don't know much about Meteor to be honest. But the fact it manages client, server, and database afraid me. I'm not a big fan of the all inclusive framework, it's intrusive and you're highly coupled to it. You can ship extremely fast with it but it was not my need for this project.

Sails is a great configuration-oriented framework. Very productive as well but when you choose this framework, you make a sails app, not a node app.

I see it like a big factory working for you, helping you to build your app over configurations. I'm also not a big fan of waterline, the embedded ORM and prefer native database adapters. It's a really nice framework but not matching with my current needs.

I quickly liked the ideas of feathers: light core, easy integration, extensible with plugins. A highly modular, loosely coupled framework and easy to integrate with an existing app.

Feathers was what I was looking for. I quickly found the plugins I needed:

feathers-hooks: a plugin for setting before/after methods hooks for validation, authorization. Here is a great article about hooks to find out more about it.

feathers-mongoose: a plugin for exposing a resource from a mongoose model

Updating the models

The answer model had no reason to exist. An answer is owned by a question and can't "live" without it. I integrated it in the question model as an embedded collection.

Here is the question model updated for the new needs

Question model updated

This way, in one atomic operation with mongoDB, I get all the data I need for a question without populating any field (answers are embedded).

Using hooks

After switching to feathers I had to make some minor changes to my Question controller as well. I directly use a service created from my question model to expose my resource though the path /api/questions.

Question controller updated with hooks

We see here how hooks are used. For example, when we add an answer to a question, it concerns the addToCollection method (a feathers plugin is currently being developed for automatically handling embedded collection in a resource).

Before adding an answer, I set the author (the current user)

After adding it, I add the author to the question's followers (author of others answers of this question) and I notify the others followers about this new answer

In order to make sure that we always have the current user available in the hooks methods, I use the following configuration (note the last .use() call):

A new start

At the beginning, my Question controller was bloated, with lots of things going on (not KISS). My database was handling lots of requests that could be avoided. And lots of code was duplicated between my resources, essentially the CRUD operations with mongoose (not DRY).

Now, my app is simple to read and modular so it's really easy to develop new features since it's a question of adding a hook or altering an existing one. I really like this aspect approach brought by hooks, it makes the app much more adaptable.

Having your Mongoose Models exposed as api resources within minutes is really awesome. You won't go back after trying this out :)