When we initially launched our API v1, we knew that it was just the first step towards our vision of an extensive domain management automation API to support our platform for DNS and domain automation. API v1 was designed as a bridge between the original API v0 and a new set of redesigned API. With years of API v1 under the belt, we had a clear picture of what worked and what didn't, and we were ready to start working on a new, completely redesigned API.

We decided to not evolve that system, but to start fresh with a new architecture.

API v1 was built with a classic Ruby on Rails app: complex business logic spread across models and controllers, rendering logic delegated to respond_to , and ActiveRecord serializers. This structure was inherited from API v0, where we adopted the standard Rails way of serving different responses from the same controller that was quite common back in 2010.

This structure, however, introduces several maintenance and scalability issues. Here are some problems we found and how we solved them.

Chaotic Code Organization

Without a clear guidance from the framework, and with no explicit team guidelines, the code for complex use cases was randomly scattered across controllers and models. The domain registration system had a different organization from the domain renewal, which had a different system than domain transfers, and so on. As you can imagine, this made maintenance hard in a large codebase like ours.

Here's an example of how the registrations controller used to be back in 2011:

class DomainRegistrationsController < ApplicationController helper :domains before_filter :require_user before_filter :setup_name_and_domain before_filter :require_subscription before_filter :setup_number_of_years before_filter :setup_registration_price def setup_name_and_domain # ... end protected :setup_name_and_domain def setup_number_of_years # ... end protected :setup_number_of_years def setup_registration_price # ... end protected :setup_registration_price # TODO: refactor this and update - duplicate code def create # ... end private def process_registration ( payment_type ) # ... end def register if @domain . register ( @number_of_years , params [ :extended_attribute ]) if @domain . registering? DomainNotifier . domain_registering ( @domain ). deliver elsif ! @domain . registered? DomainNotifier . failed_domain_registration ( @domain ). deliver end # TODO: if the domain is registering, then what happens here? if params [ :privacy ] == '1' if @wpps_supported @domain . enable_wpps DomainNotifier . whois_privacy_protection_purchased ( @domain ). deliver else DomainNotifier . whois_privacy_protection_unavailable ( @domain ). deliver end end # TODO: if the domain is registering and fails then this will result # in an offering being used with no domain delivered. @offer . use! if @offer respond_to do | format | format . html { redirect_to domains_path } format . json { render :json => @domain . to_json ( :except => 'powerdns_domain_id' ), :location => domain_path ( @domain ), :status => :created } format . xml { render :xml => @domain . to_xml ( :except => 'powerdns_domain_id' ), :location => domain_path ( @domain ), :status => :created } end else @domain . registration_failed! respond_to do | format | format . html { render :action => 'new' } format . json { render :json => { :name => @domain . name , :errors => @domain . errors . full_messages }, :status => 422 } format . xml { render :xml => { :name => @domain . name , :errors => @domain . errors . full_messages }, :status => 422 } end end end # TODO: refactor! def setup_contact if @domain . registrant == nil if params [ :contact ] base_attributes = { :user_id => current_user . id , :state_province_choice => 'S' } base_attributes [ :email_address ] = current_user . email if params [ :contact ][ :email_address ]. blank? contact_attributes = base_attributes . merge ( params [ :contact ]) @contact = @domain . build_registrant ( contact_attributes ) unless @contact . save respond_to do | format | format . html { render :action => 'new' } format . json { render :json => { :name => @domain . name , :errors => @domain . errors . full_messages + @contact . errors . full_messages }, :status => 422 } format . xml { render :xml => { :name => @domain . name , :errors => @domain . errors . full_messages + @contact . errors . full_messages }, :status => 422 } end return false end elsif current_user . default_contact @domain . registrant = current_user . default_contact else @domain . errors . add ( :base , "A registrant is required" ) respond_to do | format | format . html { render :action => 'new' } format . json { render :json => { :name => @domain . name , :errors => @domain . errors . full_messages }, :status => 422 } format . xml { render :xml => { :name => @domain . name , :errors => @domain . errors . full_messages }, :status => 422 } end return false end end logger . info "Contact setup complete" return true end # ... end

Pretty messy, isn't it?

As preparatory work for the new API, we extracted the code from models and controllers into objects we called commands. We picked this name as we originally thought we would implement the command design pattern, but it never happened and the name never changed. Today, we would probably call them interactors or operations.

These objects implement the common logic for each use case in our system. We have one object for the login use case, one for the forgotten password, one for the signup, etc.

This abstraction alone simplifies a lot the maintenance.

All the code is predictably organized.

class DomainRegistrationsController < ApplicationController # ... def create @result = DomainRegisterCommand . execute ( command_context , this_account , domain_params , contact_params ) @domain = @result . domain if @result . successful? && @result . processing? create_registering elsif @result . successful? create_successful else create_failed end end end

class DomainRegisterCommand include Command :: Command def execute ( account , domain_name , domains_params , contact_params ) # ... end private def register ( domain , extended_attributes , registrar_premium_price ) # ... end end

We didn't removed the complexity of this workflow, we just split it in smaller manageable objects. Now testing a feature is easier as we can focus only on the behavior of a single command object at the time.

In 2014 we also started to experiment our own version of service objects, that we formalized in 2016.

If you want to learn more about the evolution of our Rails architecture you can check out the slides and video of the talk Developing and maintaining a platform with Rails and Hanami that Simone presented at RailsConf 2016.

Tight Coupling

In Rails, when an action uses respond_to , the context for the web UI and the JSON API is the same: they share the same set of instance variables used for rendering.

That means you can't easily change one of these instance variables for a component (eg the UI), without affecting the behavior of the other component (eg API). Consider that a public JSON API is versioned, so you can't change it without breaking the entire ecosystem. We soon reached a state of code rigidity, where it was hard or impossible to modify the code for certain actions.

class RecordsController < ApplicationController # ... def create @result = RecordCreateCommand . execute ( command_context , @domain . zone , param_record_type , record_params ) @record = @result . data respond_to do | format | if @result . exists? format . html { redirect_to domain_records_url ( @domain ), notice: @result . warning } format . json { render json: { message: @result . warning }, status: 400 } elsif @result . successful? format . html { redirect_to domain_records_url ( @domain ) } format . json { render json: @record , status: 201 } else prepare_complex_records ( @record , param_record_type ) format . html { render_new } format . json { render json: { message: @result . error , errors: @record . errors }, status: 400 } end end end end

The most obvious solution to reduce complexity was to remove responsibilities from these actions. That's why we decided to implement the new API as a standalone Hanami application mounted inside Rails. A change in the web UI (Rails) isn't reflected in the API (Hanami) and viceversa.

When we'll sunset, API v1 we can simplify that action to:

class RecordsController < ApplicationController # ... def create @result = RecordCreateCommand . execute ( command_context , @domain . zone , param_record_type , record_params ) @record = @result . data respond_to do | format | if @result . exists? redirect_to domain_records_url ( @domain ), notice: @result . warning elsif @result . successful? render json: @record , status: 201 else prepare_complex_records ( @record , param_record_type ) render_new end end end end

With the command objects already in place, it became much simpler to implement the new endpoints of the API. Each new endpoint is implemented with an action. It has the role of accepting the input and invoking the command related to the current use case.

Here's an API v2 action:

module Api::V2 module Controllers::ZonesRecords class Create include Hanami :: Action def call ( params ) @zone = DomainFinder . find! ( params [ :zone_id ], authentication_context ). zone @result = RecordCreateCommand . execute ( command_context , @zone , params [ :type ], ZoneRecordParams . new ( params , [ :regions ])) @record = @result . data if @result . successful? && ! @result . exists render Serializers :: RecordSerializer . new ( @record ), 201 elsif @result . exists error ( 400 , I18n . t ( "api.zone_records.already_exists" )) else render Serializers :: ErrorSerializer . new ( @result . error , @record ), 400 end end end end end

It's still complex, but it has a high cohesion. The purpose is to not eliminate the complexity of the model domain (cause you can't), but to make it maintanable.

Implicit Serializations

The way ActiveRecord serializes a model into JSON is straightforward: it dumps all the attributes. Again, this is easy, but as before, it comes with a cost: you can't change a database column without breaking the JSON API backwards compatibility.

To solve this problem we had to introduce a new layer of serializers. They have the role of translating ActiveRecord attributes into a stable set of key/value pairs for JSON.

In DNSimple we have the concept of contact, which is a person or a company who registers a domain. For a contact serialization we return the email information in the payload, but the corresponding database column is email_address . The ContactSerializer helps to resolve this mismatch with a stable public name: the email key.

module Api::V2 module Serializers class ContactSerializer < ElementSerializer attributes :id , :account_id , :label , :first_name , :last_name , :job_title , :organization_name , :email , :phone , :fax , :address1 , :address2 , :city , :state_province , :postal_code , :country , :created_at , :updated_at def email object . email_address end end end end

Even a value can be subject to accidental changes. At one point, Ruby changed the way that JSON.generate serializes timestamps. Without an explicit control on the timestamps format, a change in Ruby (or another library), can suddenly change the output of the API.

require 'json' # Ruby 1.9 JSON . generate ( time: Time . now ) # => "{\"time\":\"Tue Jan 17 10:25:37 +0100 2017\"}" # Ruby 2.0 JSON . generate ( time: Time . now ) # => "{\"time\":\"2017-01-17 10:25:37 +0100\"}"

Conclusion

While the API v1 implementation had a short "time to market", it hid unfortunate surprises down the road. In order to make it easier to evolve your systems, it is good to remember to create team guidelines for code structure, avoid tight coupling with the current framework, and take control of your code via explicitness of intents.

This post only scratches the surface of the code changes behind API v2, and the reasons why we went through this journey. You can read more about the changes in API v2 in the API v2 announcement post. I also encourage you to check out the talk The Great 3 Year API Redesign that Anthony presented at CodeMash 2017, where he went through the story of the DNSimple API v2, explaining some of the decisions we took over the last 3 years that shaped the development of the API as well as challenges we faced.

Share on Twitter and Facebook