Just over two years ago, Lawrence wrote about Coach, our open-source Ruby library which makes it easier to build robust, maintainable and well-tested APIs by replacing Rails controllers built with ActionController with chains of "middleware".

Since then, we’ve continued using Coach and we’ve no doubt that it has allowed us to move fast, write code that stands the test of time and maintain developer happiness.

In this post, we'll build a simple API using ActionController, discover the pain points, and then see how Coach can help.

When good controllers go bad

We can get the hang of how Coach works by writing a simple controller-based API, and then rewriting it using Coach. Let's build an API for viewing attendees for an event, with the attendee to be returned specified by its ID:

class AttendeesController < ApplicationController rescue_from ActiveRecord : : RecordNotFound , with : :not_found_error def show attendee = Attendee . find ( params [ :id ] ) render json : attendee end private def not_found_error render json : { error : I18n . translate ( "not_found" ) } , status : 404 end end

We'd have something like this in our config/routes.rb , thereby exposing our new API at /attendees/:id :

resources :attendees , only : :show

Let's say we were going to add the ability to also update and delete attendees. These, like our #show action, would also have to load an attendee by its ID. To make our code more DRY (Don't Repeat Yourself), we can abstract the loading of the resource away, using a before_action to run some shared code before several controller actions:

before_action :find_attendee , only : [ :show , :update , :delete ] def show render json : @attendee end private def find_attendee @attendee = Attendee . find ( params [ :id ] ) end

That's not too painful. Let's say we now wanted to authenticate these API requests, with a user identifying themselves by providing an access token. We'd probably end up with something like this:

before_action :check_authorization_header before_action :check_access_token before_action :check_user_permissions private def check_authorization_header header_value = request . headers [ "HTTP_AUTHORIZATION" ] return missing_access_token_error unless header_value . present ? token_type , token = header_value . split ( " " , 2 ) return missing_access_token_error unless token_type == "bearer" @access_token = token end def check_access_token @user = User . find_by ( access_token : @access_token ) return invalid_access_token_error unless @user . present ? end def check_user_permissions unless @user . permissions . include ? ( "attendees" ) return invalid_permissions_error end end def missing_access_token_error render json : { error : I18n . translate ( "missing_access_token" ) } , status : 401 end def invalid_access_token_error render json : { error : I18n . translate ( "invalid_access_token" ) } , status : 401 end def invalid_permissions_error render json : { error : I18n . translate ( "invalid_permissions" ) } , status : 403 end

Now, as you can imagine, we'd probably want to authenticate all of our API this way. Things get a little painful when we want to share this authentication logic between controllers, and not just between actions in one controller.

We could put our new methods in ApplicationController , or even in an ActiveSupport::Concern which we include in each controller where we want it.

Whichever way we go, there are problems:

It's hard to think of a good way to customise the permissions required on a per-action basis (here, we check for the :attendees permission, but what if we wanted to change this per-controller or even per-action?)

permission, but what if we wanted to change this per-controller or even per-action?) Testing our authentication logic by itself is difficult - if we extract it into a ActiveSupport::Concern , there are ways of testing, but it's painful to set up a realistic state for our tests

, there are ways of testing, but it's painful to set up a realistic state for our tests The amount of state we have - stored in instance variables - is growing and is difficult to reason about or manage (for example, it's hard to be sure that the state you're expecting is actually set and isn't just nil )

Let's add some more complexity. Naturally, our event is very cosmopolitan, so we need to be able to return our errors in the user's language so they get the best experience. Let's add that:

around_action :with_locale def with_locale I18n . with_locale ( request . headers [ "HTTP_ACCEPT_LANGUAGE" ] ) { yield } end

As you can see, before too long:

We have a bunch of state stored in instance variables ( @user , @attendee ) which is difficult to reason about and track across the controller

, ) which is difficult to reason about and track across the controller We lack any good way to abstract away and reuse shared logic, let alone unit test it independently or configure it where required

Our controller has lots of before_action s and even around_action s which are hard to get your head around (not least because of the except s and only s)

Here's a real-life example from one of our controllers, before we migrated it to Coach. This is the logical conclusion of working with ActionController where you're doing something even mildly complex:

around_action :with_locale , except : [ :cancel ] before_action :only_allow_html , only : [ :authorize ] before_action :build_oauth_params_from_params , only : [ :authorize ] before_action :build_oauth_params_from_session , except : %i[authorize cancel] before_action :set_instance_variables before_action :check_client_id before_action :check_redirect_uri before_action :check_scope before_action :check_response_type before_action :redirect_to_cancel , except : %i[authorize cancel] before_action :persist_oauth_params_to_session , only : [ :authorize ] before_action :set_permitted_params

We've put together an example project, where you can see the final version of the controller we built, AttendeesController , plus tests.

How Coach can help

We've built a simple API and have seen the problems you can easily run into, even with very minimal complexity. Let's rewrite what we've just written using Coach, and see how it can help - we'll quickly see the benefits.

1. Rewriting our controller action as a single Coach::Middleware

Let's start by writing a single Coach::Middleware that does everything we need for this endpoint. After that, we'll begin pulling out reusable parts (e.g. authentication) and learning how to test them independently, and then put them together to form our API.

We'll refer to this middleware, the "primary" or top-level one for the API endpoint, as an action middleware to avoid things getting confusing when we build more middleware later. In Coach, everything we build is a Coach::Middleware , but middleware can depend on other middleware, making it easy to compose more complex behaviours.

module Routes module SingleCoachMiddleware module Attendees class Show < Coach : : Middleware def call I18n . with_locale ( supplied_locale ) do unless access_token && bearer_token ? return missing_access_token_error end user = User . find_by ( access_token : access_token ) return invalid_access_token_error unless user . present ? unless user . permissions . include ? ( "attendees" ) return invalid_permissions_error end attendee = Attendee . find_by ( id : request . params [ "id" ] ) return not_found_error unless attendee . present ? [ 200 , Constants : : Api : : RESPONSE_HEADERS , [ attendee . to_json ] ] end end private def supplied_locale request . headers [ "Accept-Language" ] end def access_token authorization_header & . split ( " " , 2 ) & . second end def bearer_token ? authorization_header & . split ( " " , 2 ) & . first == "Bearer" end def authorization_header request . headers [ "Authorization" ] end def invalid_access_token_error [ 401 , Constants : : Api : : RESPONSE_HEADERS , [ { error : I18n . translate ( "invalid_access_token" ) } . to_json ] , ] end def missing_access_token_error [ 401 , Constants : : Api : : RESPONSE_HEADERS , [ { error : I18n . translate ( "missing_access_token" ) } . to_json ] , ] end def invalid_permissions_error [ 403 , Constants : : Api : : RESPONSE_HEADERS , [ { error : I18n . translate ( "invalid_permissions" ) } . to_json ] , ] end def not_found_error [ 404 , Constants : : Api : : RESPONSE_HEADERS , [ { error : I18n . translate ( "not_found" ) } . to_json ] , ] end end end end end

So we can see the benefits as clearly as possible, let's write some tests. We'll use FactoryGirl to set up our models:

RSpec . describe Routes : : SingleCoachMiddleware : : Attendees : : Show do subject ( :instance ) { described_class . new ( context ) } let ( :attendee ) { FactoryGirl . create ( :attendee ) } let ( :user ) { FactoryGirl . create ( :user , :can_manage_attendees ) } let ( :context ) do { request : instance_double ( ActionDispatch : : Request , params : params , headers : headers , ) , } end let ( :params ) { { "id" = > attendee . id } } let ( :headers ) { { "Authorization" = > "Bearer #{ user . access_token } " } } describe "#call" do it { is_expected . to respond_with_status ( 200 ) } it do is_expected . to respond_with_body_that_matches ( attendee . to_json ) end context "with an invalid access token" do let ( :headers ) do { "Authorization" = > "Bearer not_a_real_token" } end it { is_expected . to respond_with_status ( 401 ) } it do error = { error : "Invalid access token" } is_expected . to respond_with_body_that_matches ( error . to_json ) end context "specifying a non-default language" do before { headers [ "Accept-Language" ] = "fr" } it do error = { error : "Jeton d'accès invalide" } is_expected . to respond_with_body_that_matches ( error . to_json ) end end end context "with a user who doesn't have access" do let ( :user ) { FactoryGirl . create ( :user ) } it { is_expected . to respond_with_status ( 403 ) } it do error = { error : "Invalid permissions" } is_expected . to respond_with_body_that_matches ( error . to_json ) end end context "with a non-existent attendee" do let ( :params ) { { id : "not_a_real_id" } } it { is_expected . to respond_with_status ( 404 ) } it do error = { error : "Not found" } is_expected . to respond_with_body_that_matches ( error . to_json ) end end end end

Finally, we can link our action up to the Rails router in config/routes.rb like this:

router = Coach : : Router . new ( self ) router . draw ( Routes : : SingleCoachMiddleware : : Attendees , base : "single_coach_middleware/attendees" , actions : [ :show ] )

2. Translating errors into the requested language

We now have a simple and fully-tested API. However, our middleware's #call method has a tonne of responsibilities, and we can't test anything in isolation or reuse our logic.

Let's start by extracting out the multi-language behaviour into a discrete Coach::Middleware , which our action will declare as a dependency:

module Middleware class Translate < Coach : : Middleware def call I18n . with_locale ( supplied_locale ) { next_middleware . call } end private def supplied_locale request . headers [ "Accept-Language" ] end end end

Here, we define a #call method as before - this is what all Coach::Middleware look like. The one thing that changes in this one is that we do a single, discrete job, and then call next_middleware.call when we're done, rather than returning a response. This is similar to Rack middleware, which are at the heart of how Rails itself works.

In this case, we open an I18n.with_locale block, and then call our next middleware from within that block. This means that our calls to I18n.translate in our action will return strings in the requested language.

We can unit test Middleware::Translate independently of any specific action middleware, like this:

`spec / middleware / translate_spec . rb` RSpec . describe Middleware : : Translate do subject ( :instance ) { described_class . new ( context , next_middleware ) } let ( :context ) do { request : instance_double ( ActionDispatch : : Request , headers : headers ) } end let ( :headers ) { { "Accept-Language" = > "fr" } } context "with a language set" do let ( :next_middleware ) { - > { expect ( I18n . locale ) . to eq ( :fr ) } } it "sets the language" do instance . call end end context "with no language set" do let ( :headers ) { { } } let ( :next_middleware ) { - > { expect ( I18n . locale ) . to eq ( :en ) } } it "uses the default language" do instance . call end end end

How does this end up working? We can now start using this in Routes::Attendees::Show , which will make things much clearer:

module Routes module SingleCoachMiddleware module Attendees class Show < Coach : : Middleware uses Middleware : : Translate def call unless access_token && bearer_token ? return missing_access_token_error end user = User . find_by ( access_token : access_token ) return invalid_access_token_error unless user . present ? unless user . permissions . include ? ( "attendees" ) return invalid_permissions_error end attendee = Attendee . find_by ( id : request . params [ "id" ] ) return not_found_error unless attendee . present ? [ 200 , Constants : : Api : : RESPONSE_HEADERS , [ attendee . to_json ] ] end private end end end end

We call the uses method in our action to declare the Translate middleware as a dependency. Middleware we declare with uses get run in order before our action. So Middleware::Translate will get run first, and it is responsible for calling our action, the next middleware in the chain (hence the next_middleware.call ).

We've now extracted the ability to set the language into its own middleware and tested that behaviour on its own, so we can remove tests related to that from the action middleware's unit tests (which we wrote in step #1).

It's worth nothing that we may well want to add integration tests, exercising the route and all the middleware it depends on, on top of our unit tests - but even there we won't test every possible case, instead focusing on the most common paths, trusting our rigorously unit-tested middleware to work (a bit like we trust the tested behaviours of Rails to just work).

3. Authenticating with an access token

Next, let's try doing the same with our authentication functionality, splitting it out into a middleware:

module Middleware class Authenticate < Coach : : Middleware provides :user def call return missing_access_token_error unless access_token . present ? user = User . find_by ( access_token : access_token ) return invalid_access_token_error unless user . present ? provide ( user : user ) next_middleware . call end private def access_token bearer_token ? && authorization_header & . split ( " " , 2 ) & . second end def bearer_token ? authorization_header & . split ( " " , 2 ) & . first == "Bearer" end def authorization_header request . headers [ "Authorization" ] end def invalid_access_token_error [ 401 , Constants : : Api : : RESPONSE_HEADERS , [ { error : I18n . translate ( "invalid_access_token" ) } . to_json ] , ] end def missing_access_token_error [ 401 , Constants : : Api : : RESPONSE_HEADERS , [ { error : I18n . translate ( "missing_access_token" ) } . to_json ] , ] end end end

This middleware is similar to our last one, but with two major differences:

In certain situations, it returns a HTTP response, rather than calling next_middleware.call .

. We "provide" data. This replaces how we typically use instance variables in controllers. We define on the middleware class what data it can be expected to "provide", and then we "provide" it. This can be accessed by other middlewares, including our action, which "require" it.

We can access the user value which was provided by Middleware::Authenticate in our action. Explicitly defining these requires and provides allows Coach to perform some helpful error checking. If we "require" a certain piece of context, but omit the middleware which "provides" it, Coach will throw an exception when the application boots.

Let's add some tests for our new middleware. Coach provides some helpful RSpec matchers, like respond_with_status , respond_with_body_that_matches and provide to make it easy to write clean, readable tests. To make these available, you just need to require 'coach/rspec' (most likely in your spec/rails_helper.rb file).

RSpec . describe Middleware : : Authenticate do subject ( :instance ) { described_class . new ( context , null_middleware ) } let ( :context ) do { request : instance_double ( ActionDispatch : : Request , headers : headers ) , } end let ( :headers ) do { "Authorization" = > authorization_header , } end context "with a valid Authorization header" do let ( :user ) { FactoryGirl . create ( :user ) } let ( :authorization_header ) { "Bearer #{ user . access_token } " } it { is_expected . to call_next_middleware } it { is_expected . to provide ( user : user ) } end context "with an Authorization header with an invalid bearer token" do let ( :authorization_header ) { "Bearer lol" } it { is_expected . to respond_with_status ( 401 ) } it do is_expected . to respond_with_body_that_matches ( /Invalid access token/ ) end end end

Let's start using our newly-provided user in our action:

module Routes module SingleCoachMiddleware module Attendees class Show uses Middleware : : Translate uses Middleware : : Authenticate requires :user def call unless user . permissions . include ? ( "attendees" ) return invalid_permissions_error end attendee = Attendee . find_by ( id : request . params [ "id" ] ) return not_found_error unless attendee . present ? [ 200 , Constants : : Api : : RESPONSE_HEADERS , [ attendee . to_json ] ] end private end end end end

4. Checking users' permissions

We're still checking the user's permissions in our action. Let's abstract that out and make it reusable, writing another middleware:

module Middleware class CheckUserPermissions < Coach : : Middleware requires :user def call unless user . permissions . include ? ( scope ) return invalid_permissions_error end next_middleware . call end private def scope config . fetch ( :scope ) . to_s end def invalid_permissions_error [ 403 , Constants : : Api : : RESPONSE_HEADERS , [ { error : I18n . translate ( "invalid_permissions" ) } . to_json ] , ] end end end

This middleware introduces two new things:

Everything is a Coach::Middleware , so our middleware can require things (like our action requires the user provided by Middleware::Authenticate ) as well as provide them

, so our middleware can require things (like our action requires the user provided by ) as well as provide them This middleware is configurable, referring to config[:scope] , making it more abstract and reusable

We won't write tests here for this new middleware to keep things short, but you can find the tests for this middleware and the next one in our sample project.

To provide configuration for a middleware, we just pass keyword arguments to uses :

module Routes module SingleCoachMiddleware module Attendees class Show uses Middleware : : Translate uses Middleware : : Authenticate uses Middleware : : CheckUserPermissions , scope : :attendees requires :user def call attendee = Attendee . find_by ( id : request . params [ "id" ] ) return not_found_error unless attendee . present ? [ 200 , Constants : : Api : : RESPONSE_HEADERS , [ attendee . to_json ] ] end private end end end end

5. Loading the right record from the database

Finally, let's move our shared logic for finding the right record in the database to a middleware. Since this is likely to be a commonly-performed task with a range of models, we'll make it a bit more abstract, supporting any model:

module Middleware class GetModelById < Coach : : Middleware provides :model def call model = model_class . find_by ( id : id ) return not_found_error unless model . present ? provide ( model : model ) next_middleware . call end private def model_class config . fetch ( :model_class ) end def id request . params [ "id" ] end def not_found_error [ 404 , Constants : : Api : : RESPONSE_HEADERS , [ { error : I18n . translate ( "not_found" ) } . to_json ] , ] end end end

Once again, our middleware is configurable, so we'll pass the model_class keyword argument when we add it to the chain by calling uses :

module Routes module CoachMiddlewareChain module Attendees class Show < Coach : : Middleware uses Middleware : : Translate uses Middleware : : Authenticate uses Middleware : : CheckUserPermissions , required_permission : :attendees uses Middleware : : GetModelById , model_class : Attendee requires :user , :model def call [ 200 , Constants : : Api : : RESPONSE_HEADERS , [ model . to_json ] ] end private end end end end

6. Writing a unit test for our action middleware

We've now moved from a complicated controller to a chain of modular Coach::Middleware , each with a single responsibility.

Our shared middleware are unit-tested, but we don't have any unit tests covering the action itself, or an integration test covering the whole flow from top to bottom. Let's add a unit test so we can be confident our API works as expected.

The unit test for our action will make sure that, using the inputs provided by our previous middleware, our action produces the right result. This looks very simple, with just a single example, since the complicated logic is handled by individually-tested middleware which have been abstracted away:

RSpec . describe Routes : : CoachMiddlewareChain : : Attendees : : Show do subject ( :instance ) { described_class . new ( context ) } let ( :attendee ) { FactoryGirl . create ( :attendee ) } let ( :user ) { FactoryGirl . create ( :user , :can_manage_attendees ) } let ( :context ) do { request : instance_double ( ActionDispatch : : Request ) , model : attendee , user : user , } end describe "#call" do it { is_expected . to respond_with_status ( 200 ) } it do is_expected . to respond_with_body_that_matches ( attendee . to_json ) end end end

7. Finishing off with an integration test

The integration test is a little more involved. Here, we'll simulate a real HTTP request coming in, and make sure that the whole chain of middleware does what we expect, returning the correct response.

We don't need to test every single case, since we should be able to trust the unit tests of our middleware, but here for completeness, we'll do a fairly thorough test. As you write further actions for your API, you'll develop an intuition and conventions to help you decide what to test.

RSpec . describe "GET /single_coach_middleware/attendees/:id" , type : :request do let ( :attendee ) { FactoryGirl . create ( :attendee ) } let ( :user ) { FactoryGirl . create ( :user , :can_manage_attendees ) } let ( :headers ) do { "Accept" = > "application/json" , "Authorization" = > "Bearer #{ user . access_token } " , } end it "returns a JSON representation of the attendee" do get "/single_coach_middleware/attendees/ #{ attendee . id } " , headers : headers expect ( response . status ) . to eq ( 200 ) expect ( response . body ) . to eq ( attendee . to_json ) end context "requesting a non-existent ID" do it "returns an error" do get "/single_coach_middleware/attendees/123" , headers : headers expect ( response . status ) . to eq ( 404 ) expect ( response . body ) . to eq ( { error : "Not found" } . to_json ) end end describe "authentication" do context "with a user with insufficient permissions" do let ( :user ) { FactoryGirl . create ( :user ) } it "returns an error" do get "/single_coach_middleware/attendees/ #{ attendee . id } " , headers : headers expect ( response . status ) . to eq ( 403 ) expect ( response . body ) . to eq ( { error : "Invalid permissions" } . to_json ) end end context "not providing an Authorization header" do before { headers . delete ( "Authorization" ) } it "returns an error" do get "/single_coach_middleware/attendees/ #{ attendee . id } " , headers : headers expect ( response . status ) . to eq ( 401 ) expect ( response . body ) . to eq ( { error : "Missing access token" } . to_json ) end end context "providing an invalid access token" do before { headers [ "Authorization" ] = "Bearer lolwut" } it "returns an error" do get "/single_coach_middleware/attendees/ #{ attendee . id } " , headers : headers expect ( response . status ) . to eq ( 401 ) expect ( response . body ) . to eq ( { error : "Invalid access token" } . to_json ) end context "requesting errors in French" do before { headers [ "Accept-Language" ] = "fr" } it "returns an error in French" do get "/single_coach_middleware/attendees/ #{ attendee . id } " , headers : headers expect ( response . status ) . to eq ( 401 ) expect ( response . body ) . to eq ( { error : "Jeton d'accès invalide" } . to_json ) end end end end end

We've discovered how Coach can help us write modular, unit testable APIs that are easy to reason about.

We started with an ActionController action, depending on a bunch of before_action s and even an around_action , which was hard to test, made it tricky to pull out abstractions and was confusing for the developer.

We rewrote that controller action as a single Coach action, encapsulated in a Coach::Middleware and called from our Rails router. We wrote a single unit test for that action.

We then pulled our individual jobs that our action did into separate, individually-tested Coach::Middleware that our top-level "action" middleware then depended on. We start with building Middleware::Translate , then Middleware::Authenticate and then two configurable (and thus easily reusable) middleware, Middleware::CheckUserPermissions and Middleware::GetModelById .

You can see all these steps, complete with tests, in our demo project.

Coach encourages patterns that have scaled well with our engineering team, while new joiners frequently say Coach is their favourite part of our codebase. We'd love to see other teams enjoy the benefits we've already gained from Coach.

Building a world-class payments API is at the heart of what we do at GoCardless, and Coach has been one of the key tools we’ve used (and built!) to maintain simplicity in our codebase as we strive towards this goal.