Using Foreman with service oriented architecture

I’ve been working with some friends on a fun side-project lately - AppShape. It’s a product that will allow users to constantly monitor their applications health from both technical and business perspective.

We decided to split the product into 3 separate applications

appshape - user facing web application running on Ruby On Rails

- user facing web application running on Ruby On Rails appshape-runner - runs user-defined checks against their applications, written in Ruby

- runs user-defined checks against their applications, written in Ruby appshape-reporter - reports results of this checks back to main application, written in Ruby

At the moment we are also using two beanstalkd queues and clockwork process to glue everything together.

This means launching 6 processes in 6 different terminal windows, constantly switching between them to check logs…

I’ve had to come up with something better.

Foreman

I like the simplicity of the Procfile approach provided by Foreman. It’s just one command and you get colorized output, all logs in one window and ability to kill all your processes with CTL+C . I wanted that for my project!

Unfortunately Foreman doesn’t support the way we’ve structured AppShape:

it doesn’t handle running processes from a different directory

it doesn’t support different RVM’s which we are using for handling multiple ruby versions/gemsets

Subcontractor

After short googling I’ve found subcontractor - nice gem that is supposed to handle all of this.

Let’s take a look at Procfile with subcontractor commands included:

web: bundle exec puma -C ./config/puma.rb runner_queue: beanstalkd -p 11305 reporter_queue: beanstalkd -p 11306 clock: clockwork clock.rb runner: subcontract --rvm 'ruby-2.2.3@appshape-runner' --signal TERM --chdir ../appshape-runner -- bundle exec ruby main.rb reporter: subcontract --rvm 'ruby-2.2.3@appshape-results-reporter' --signal TERM --chdir ../appshape-reporter -- bundle exec ruby main.rb

The usage is pretty simple

--rvm - sets ruby version and gemset

- sets ruby version and gemset --signal - which signal should be send to the process

- which signal should be send to the process --chdir - to where change directory

- to where change directory -- bundle exec ruby main.rb - command that should be executed

With this setup I was able to start foreman as usual with foreman start and I could see all my processes running.

I would be perfectly happy with the above but I’ve noticed that sometimes processes started via subcontractor were not killed. After few hours of work and multiple restarts I executed ps aux | grep ruby and saw about 20 running ruby processes.

I haven’t been able to fix the issue in a reasonable amount of time so I’ve decided to try another approach.

Foreman (again)

Back in the old days I was using bash -lc when I wanted to run some of my ruby programs via cron . It starts non-interactive login shell and executes given command. The trick here is that login option. Amongst other things it will source your .profile files so RVM will be available!

Here’s my modified Procfile

web: bundle exec puma -C ./config/puma.rb runner_queue: beanstalkd -p 11305 reporter_queue: beanstalkd -p 11306 clock: clockwork clock.rb runner: bash -lc "cd ../appshape-runner && bundle exec ruby main.rb" reporter: bash -lc "cd ../appshape-reporter && bundle exec ruby main.rb"