At FireHydrant, we built a platform for integrations from the beginning. This has a lot of implications to the way that data gets created and read out of your application however. For example, how does a user differ from an integration reading data? Do you make API tokens on users or create a separate model called Bot ? (Hint: you do). These questions are something that all engineering teams need to solve if you’re building a B2B SaaS product that others can also integrate with. In this blog post we’ll talk about how FireHydrant is built to support creating data in our integration ready platform.

Data Model

One of the first things that makes a B2B company different when getting started is the initial data model. Traditionally you read tutorials on how to make a user based data model (users signup and use your services). These lessons change, though, when you introduce the concept of a company signing up for your services and adding users. For example, your company might be using Google for your company email. You as the individual are not paying for it, your company is, so how do you build an application that supports this model?

For FireHydrant, we have 4 models to enable this idea:

Account

Organization

User

OrganizationUser

In this model you can see that Account sits at the top and everything has an account_id reference to it (more on this later). Account’s also have many Organizations, and Organization has many users through the OrganizationUser model.

Account is a model that is where all of the high level detail about a customer exists, remember, our application is for businesses and their users, not individuals only. By moving a model above everything called “Account” we can put billing information, addresses, roles, etc, at the top. In our app, Account is more or less a “god model”, but it has no responsibility other than to be a place everything can point to.

The other model you might see here and go “wtf mate?” Is the Organization model. The reason we have this model in our stack is to support very large organizations that have several divisions that have hundreds of users in each. If you think about a company like Microsoft, you might have an Organization for Azure, and another for Office365, but the account is still just “Microsoft”.

Our models are represented like so:

class Account < ActiveRecord::Base has_many :users has_many :organizations validates :name, presence: true end class Organization < ApplicationRecord has_many :organization_users, dependent: :destroy has_many :users, through: :organization_users, dependent: :destroy end class OrganizationUser < ApplicationRecord belongs_to :user belongs_to :organization end class User < ApplicationRecord has_many :organization_users, dependent: :destroy has_many :organizations, through: :organization_users end

The astute reader might have noticed that every model here inherits from the ApplicationRecord class except for the Account model.

The reason for this is because we have a non-nullable account_id on every model in our stack (minus the Account model). We then have this ApplicationRecord class definition:

class ApplicationRecord < ActiveRecord::Base self.abstract_class = true belongs_to :account end

This forces all of our models to have an association to the account record that owns that piece of data in our database.

Now that we have a model that allows us to create data in a tenanted fashion, let’s get on to the juicy bits.

Creating Data

We’re an incident response tool, so naturally we have a model in our Rails application called Incident , but incidents can be created by people, bots, and integrations. In most Rails applications where data is created by a user, you’ll typically see a belongs_to :user line in the model. This is so you can see who authored (or owns) that record. For us, though, this doesn’t work when you have multiple types of actors that can create data.

To solve this, we have an object pattern called Creators in our application. The idea of a creator is that is handles all of the logic necessary to create data. Rails typically recommends having a controller action create data with your model class directly, for us, though, this simply isn’t practical for how many different ways data can be created.

For example, our PostMortem::Report model has this definition:

class PostMortems::Report < ApplicationRecord validates :name, presence: true belongs_to :created_by, polymorphic: true belongs_to :incident end

Notice how our created_by association is marked as polymorphic. This means we can associate any model on our PostMortem::Report as the actor that created it. This was is a weird thing to start doing but has made our application so flexible for integrations I wouldn’t do it any other way now.

But creating these objects can be weird, since an API endpoint, controller action, or even just a Sidekiq job needs a lot more context now. This is where our creator pattern emerged.

An example of one of our production creator objects is for postmortem reports:

creator = ::PostMortems::ReportCreator.new ( incident, actor: current_user, account: current_account ) result = creator.create ( params )

All creators have the first parameter of “scope”. Since everything after an account belongs to something, it makes sense to have your initializer accept the thing that data will ultimately belong to when saved.

Secondly, we have 2 keyword arguments for the actor creating the data, and the account that will own it. An actor could be a User , Bot , or even Integrations::Slack::Connection object. Since the created_by field is polymorphic, we can accept any model type in this field.

Then our creator object has a #perform method that is called by our #create method you see in the example above. It looks like this:

class PostMortems::ReportCreator < ApplicationCreator private alias incident scope delegate :organization, to: :incident def perform ( params = { } ) allowed = AllowedParams.new ( params ) report = PostMortems::Report.new ( allowed.to_h ) result = WriterResult.new ( report ) PostMortems::Report.transaction do report.created_by = actor report.incident = incident report.account = account unless report.save result.append_errors ( report ) raise ActiveRecord::Rollback end end end class AllowedParams < WriterParams property :name property :summary property :tag_list end end

From the top you can see we inherit from our ApplicationCreator class. This is where #create is defined which delegates to the #perform method in this object, but wraps it logs and metrics (another huge advantage to this we’ll talk about later).

You can see we have another object called AllowedParams . Rails provides StrongParams, but what if the params didn’t come from a controller action… should we forgo all parameter sanitization because of that? Of course not! So we allow parameters in from anywhere but only pass the ones we care about to our model using this class definition.

Next, we assign the created by field, incident, and our account. From there, we attempt to save the object, and if it fails we take the errors from the report model object and append them to our WriterResult object. WriterResult is pretty simple:

class WriterResult < Struct.new ( :object ) def success? errors.empty? end alias successful? success? def errors? errors.any? end def append_errors ( object ) errors.merge ! ( object.errors ) end def errors @errors || = ActiveModel::Errors.new ( self ) end end

This object makes returning a result from a #create call easy, and allows us to return the object created or errors. ActiveRecord::Errors has a mutating method merge! that will merge other instances of the class into itself. This allows us to merge in errors from other objects into the result object.

Folder Structure

Because Rails adds all folders in the app/ directory to the auto load path, we have 3 folders for our object mutation classes:

app/creators app/updaters app/destroyers

So our postmortem creator class lives at:

app/creators/post_mortems/report_creator.rb

Usage

You can see how we use it in our PostMortems API endpoint:

post do incident = current_organization.incidents.find ( params [ :incident_id ] ) creator = ::PostMortems::ReportCreator.new ( incident, **actor_and_account ) result = creator.create ( params ) if result.success? present result.object, with: PublicAPI::V1::PostMortems::ReportEntity else status 400 present PublicAPI::V1::Error.new ( detail: "could not create report" , messages: result.errors.full_messages ) , with: PublicAPI::V1::ErrorEntity end end

Pretty simple, right? Any other entry points for creating a PostMortem::Report object are exactly the same too, creating consistency throughout the entire application.

There are other advantages of using this pattern as well. For example, permissions. Permissions can now live in our creators / updaters objects. And because we’re returning a WriterResult object, we can append any errors about permissions to that result object instead.

Telemetry

Another huge advantage of using a single object for creating any data in your application is telemetry. Logs and metrics are not as readily available for normal model creates if you’re doing controller -> model. With our creator objects, however, we can easily log and emit metrics for everything with ease.

Here is a stripped version of our ApplicationCreator class:

class ApplicationCreator def initialize ( scope, actor:, account: ) @scope = scope @actor = actor @account = account end def create ( params = { } ) perform ( params ) end private attr_reader :scope, :actor, :account end

Notice how our create method here just delegates to the #perform method on the class. Because we inherit all of our creator objects from it, we have the ability to wrap the logic with telemetry:

def create ( params = { } ) Instrumentation.counter ( :create, { creator: self.class.name } , 1 ) do perform ( params ) end end

Our instrumentation in our Rails application allows us to tag metrics in prometheus. In this instance we are counting and timing the create call for our creator class, making it easy to see how long our create calls are really taking.

Logs become extremely simple now as well:

def create ( params = { } ) Instrumentation.counter ( :create, { creator: self.class.name } , 1 ) do perform ( params ) end Rails.logger.info ( { event: "create" , creator: self.class.name, account_id: account.id, actor_type: actor.class.name, actor_id: actor.class.id } ) end

Having logs for when creators are called incredibly valuable, and we also have the ability to see the account and actor that performed the action because we enforce that all creates include that information.

Using Google PubSub in Creators

Our application makes use of Google’s PubSub product to notify other parts of our application of an event occurring. For example, when a note is added to an incident timeline, we push a message to PubSub after the note is created in our creator object. This means that anytime we create a note (via the API, UI, or Slack integration), a message is published the same exact way and all of the other systems can react accordingly.

class Incidents::NoteCreator < ApplicationCreator private alias incident scope def perform ( params ) allowed = AllowedParams.new ( params ) note = Event::Note.new ( allowed.to_h ) note.account = account result = WriterResult.new ( note ) ActiveRecord::Base.transaction do incident_event = IncidentEvent.new ( event: note, created_by: actor, incident: incident, account: account ) if params [ :visibility ] .present? incident_event.visibility = params [ :visibility ] end unless note.save && incident_event.save result.append_errors ( note ) result.append_errors ( incident_event ) return result end publisher = Incidents::EventPublisher.new ( incident ) publisher.publish ( incident_event, 'created' ) end result end class AllowedParams < WriterParams property :body end private_constant :AllowedParams end

This creator does quite a few things for us:

Creates a note record

Ties that record to an incidents timeline (That’s our IncidentEvent model)

model) Publish our incident event to PubSub

This object has a ton of specs to ensure all of these cases work as designed, it is complex, but it’s as complex as it needs to be for our uses. Because of this object though, adding a note to an incident via our Slack integration is extremely simple:

class Integrations::Slack::Commands::NewNote < Integrations::Slack::Commands::BaseCommand include Integrations::Slack::Commands::LinkedAccountCheck include Integrations::Slack::Commands::ActiveChannelCheck def to_channel_response creator = Incidents::NoteCreator.new ( current_incident, actor: authed_provider.user, account: account ) result = creator.create ( body: parsed.string_args ) errors.merge ! ( result.errors ) nil end private def parsed @parsed || = intent.parsed_original_text end end

We’ll leave some parts of this to imagination. The key point is that it’s only a few lines of code to create a note, and it’s also the same exact lines (give or take a few parameter changes) in our API too.

Breakdown

After building several Rails applications that have provide public APIs and are B2B products, I think this is the right way forward for us. We’ve been using this pattern in production for months and haven’t seen many cracks with it just yet.

We’re betting on our future with this pattern too. It is significantly more difficult to go back to every entry point of data creation/mutation to add logs, telemetry, queue publishing, etc after a few years of operation. Depending on the size of your application, you could be looking at years of dedicated work to accomplish these things.

——

I hope this was helpful, if you have any questions about the stack, our decisions, or if you want to tell me this is a horrible idea, my Twitter handle is @bobbytables.

You just got paged. Now What? Get started with FireHydrant