In a previous article I explored how I make putting work into the background easier. The goal is to be able to decide when to run some procedure immediately or to run it asynchronously via a background job. Here it is:

class SomeProcess class Later < Que::Job def run(*args) options = args.pop # get the hash passed to enqueue ::SomeProcess.new(args).send(options['trigger_method']) end end def initialize(some_id) @some_id = some_id @object = User.find(some_id) end attr_reader :some_id def later(which_method) Later.enqueue(some_id, 'trigger_method' => which_method) end def call # perform some long-running action end end

This works well for this class, but eventually we'll want to use this same idea elsewhere. You can always copy and paste, but we know that's a short term solution.

Generalizing your solution

Here's how we can take a solution like this and turn it into a more general tool.

First, I like to come up with the code that I want to write in order to use it. Deciding what code you want to write often means deciding how explicit you want to be.

Do we want to extend or include a module? How should we specify that methods can be performed later? Do we need to provide any default values?

I often begin answering these questions for myself but end up changing my answers as I think through them or even coming up with additional questions.

Here's where I might start...

Often, I want my code to clearly opt in to using a library like the one we're building. It is possible, however, to automatically make it available.

We can monkey-patch Class for example so that all classes might have this ability. But implicitly providing features to a vast collection of types lacks the clarity that developers of the future will want to find when reading through or changing our code.

Although I want to be able to make any class have the ability to run in the background, I'll want to explicitly declare that it can do that.

class SomeProcess include ProcessLater end

And here's what we would need inside that module:

module ProcessLater def later(which_method) Later.enqueue(some_id, 'trigger_method' => which_method) end class Later < Que::Job def run(*args) options = args.pop # get the hash passed to enqueue ::SomeProcess.new(args).send(options['trigger_method']) end end end

We've just moved some code around but have mostly left it the way it was before. This means we'll have a few problems.

Overcoming specific requirements in generalizations

Our ProcessLater module has a direct reference to SomeProcess so the next class where we attempt to use this module will have trouble.

We need to tell our background job what class to initialize when it's pulled from the queue.

That means our Later class needs to look something like this:

class Later < Que::Job def run(*args) options = args.pop # get the hash passed to enqueue class_to_run.new(args).send(options['trigger_method']) end end

Every class that uses ProcessLater would need to provide that class_to_run object. We could initialize our Later class with an argument, but often with background libraries we don't have control over the initialization. Typically, all we get is a method like run or perform which accepts our arguments.

We'll get to solving that in a minute but another problem we'll see is that every queued job would be for the ProcessLater::Later class. Even though we're creating a generalized solution, I'd rather see something more specific in my queue.

I like to keep related code as close together as is reasonably possible and that leads me to nesting my background classes within the class of concern.

Here's an example of what jobs I'd like to see in my queue: SomeProcess::Later , ComplexCalculation::Later , SolveHaltingProblem::Later .

Seeing that data stored for processing (along with any relevant arguments) would give me an idea of what work would need to be done.

Creating a custom general class

We can create those classes when we include our module.

module ProcessLater def later(which_method) Later.enqueue(some_id, 'trigger_method' => which_method) end class Later < Que::Job # create the class lever accessor get the related class class << self attr_reader :class_to_run end # create the instance method to access it def class_to_run self.class.class_to_run end def run(*args) options = args.pop # get the hash passed to enqueue class_to_run.new(args).send(options['trigger_method']) end end def self.included(klass) # create the unnamed class which inherits what we need later_class = Class.new(::ProcessLater::Later) # assign the @class_to_run variable to hold a reference later_class.instance_variable_set(:@class_to_run, self) # name the class we just created klass.const_set(:Later, later_class) end end

There's a lot going on there but the end result is that when you include ProcessLater you'll get a background class of WhateverYourClassIs::Later .

But there's still a problem. The ProcessLater module has our later method enqueue the background job with Later which will actually look for ProcessLater::Later but we need it to be specifically the class we just created.

We want the instance we create to know how to enqueue itself to the background. All we need to do is provide a method which will look for that constant.

module ProcessLater def later(which_method) later_class.enqueue(some_id, 'trigger_method' => which_method) end private # Find the constant in the class that includes this module def later_class self.class.const_get(:Later) end

Knowing how to initialize

There's still one problem: initializing your object.

The later method knows about that some_id argument. But not all classes are the same and arguments for initialization are likely to be different.

We're going to go with a "let's just make it work" kind of solution. Since we need to know how to initialize, we can just put those arguments into an @initalizer_arguments variable.

class SomeProcess include ProcessLater def initialize(some_id) @initializer_arguments = [some_id] @object = User.find(some_id) end attr_reader :initializer_arguments end

Now, instead of keeping track of an individual value, we track an array of arguments. We can alter our enqueueing method to use that array instead:

module ProcessLater def later(which_method) later_class.enqueue(*initializer_arguments, 'trigger_method' => which_method) end

Our general solution will now properly handle specific class requirements.

The downside with this is that we have this implicit dependency on the initializer_arguments method. There are ways around that and techniques to use to ensure we do that without failure but for the sake of this article and the goal of creating this generalized library: that'll do.

I'll cover handling those requirements like providing initializer_arguments in the future, but for now: how would you handle this? What impact would code like this have on your team?

A thin, slice between you and the background.

With that change, we're enqueueing our background jobs with the right classes.

Here's the final flow:

Initialize your class: SomeProcess.new(123) Run later(:call) on it That enqueues the details storing the background class as SomeProcess::Later The job is picked up and the SomeProcess::Later class is initalized The job object in turn initializes SomeProcess.new(123) and runs your specified method: call

That gives us a very small generalized layer for moving work into the background. What you'll see in your main class files is this:

class SomeProcess include ProcessLater def initialize(some_id) @initializer_arguments = [some_id] @object = User.find(some_id) end attr_reader :initializer_arguments def call # perform some long-running action end end

And here's the final library:

module ProcessLater def later(which_method) later_class.enqueue(initializer_arguments, 'trigger_method' => which_method) end private def later_class self.class.const_get(:Later) end class Later < Que::Job # create the class lever accessor get the related class class << self attr_accessor :class_to_run end # create the instance method to access it def class_to_run self.class.class_to_run end def run(*args) options = args.pop # get the hash passed to enqueue self.class_to_run.new(args).send(options['trigger_method']) end end def self.included(klass) # create the unnamed class which inherits what we need later_class = Class.new(::ProcessLater::Later) # name the class we just created klass.const_set(:Later, later_class) # assign the class_to_run variable to hold a reference later_class.class_to_run = klass end end

We'll explore more about building your own tools in the future and I put a lot of effort into explaining what you can do with Ruby in the Ruby DSL Handbook, so check it out and if you have any questions (or feedback), just hit reply!

Certainly some will say "Why aren't you using ActiveJob?" or "Why aren't you using Sidekiq?" or "Why aren't you ...."

All of those questions are good ones.

The way your team works, interacts, and builds their own tools has a lot more to do with answering those questions than my reasons. Many different decisions can be made but it's important for your whole team to understand which questions are the most important to answer.

Follow-up this article with the next in the series: Building a tool that's easy for your team to use