If you’ve never seen the above image before, then you can close this tab. Otherwise, I think you can find some interesting info that will help you when ‘something goes wrong’.

Background

Some time ago, I was working on an application with many forms and unfortunately, many errors. Each time I needed to reproduce an exception from the staging server, I had to send all the fields in the same way as the client.

- Can you help me reproduce this error? - Sure. You must type a text with a special character (e.g. @) in the field no. 13 and set the third dropdown to native. Oh, and remember to go to the user settings and change the target language to Chinese in the second bookmark. Then the exception will appear. - Mmm, wonderful…

The Idea

The idea of CrashBreak is to reproduce exceptions from the staging server on the programmer’s computer. It’s done by creating a test request file with the same request data and system state (database, configs, sessions). In order to do it, the library must dump your system piece after piece.

An Example

Let’s imagine a situation that ends with an error: for example, there’s a requirement that each project name on your website must start with a capital letter. Let’s write a decent code without thinking too much and mark the task as done.

def show

@project = Project.find params[:id]

@project.name[0].upcase

end

Yeah! Looks great. We can go home, but we can’t sleep peacefully. Because what will happen if the project name is empty, is, of course:

undefined method `upcase’ for nil:NilClass

And so we have a bug in our system. It’s combined with the specific request made by the user and with the state of our database. We’ll need two dumpers for the reproduction, one for each of the affected system parts.

Dumpers

A dumper is like an employee who has only one responsibility — to dump a part of the system. Dumping a user’s request is quite easy. We can easily get all the data of the request (such as URL, body, headers) to use it later for the reproduction. Unfortunately, the database is a little more problematic. Dumping a database is a time-consuming process that uses the resources of our machine. I’d suggest to use a special ‘fork’ notificator so that the entire process takes place in a separate fork.

config.exception_notifier = Crashbreak::ForkExceptionNotifier.new

This way, the error will be immediately displayed to the user.

Database dumper

The most controversial part of CrashBreak is the database dumper. As I mentioned before, dumping process can absorb resources. It’s also important to keep the dump file in a safe place. I suggest to use the database dumper exclusively on the staging server if your production database is quite big and you don’t want to increase the load of the machine working. You also need a storage server (for now CrashBreak supports AWS). After the dumping process, the data is sent to your AWS server, and the website only receives the name of the file for the future reproduction process.

Let’s Reproduce It!

Okay, we’re halfway there. Now we must download the data from CrashBreak and use it in the reproduction process of the exception. By running the rake command, you can generate a request test that will reproduce your environment during the error.

rake crashbreak:generate_test error_id=123

This is an CrashBreak test for rspec (minitest is also supported):

require 'spec_helper'



describe 'error id: 123', type: :request do

let(:restorers_data) do # data from all dumpers

Crashbreak::StateRestorer.new('123').restore

end let(:request_parser) do # parses request data

Crashbreak::RequestParser.new restorers_data[:request]

end it 'sends request' do

request_parser.request_data do |method, url, body, headers|

# e.g: get '/projects' { id: 1 }, {}

method(request_method).call url, body, headers

end

end

end

Restores

The above test goes through all the data of the dumpers and transfers it to a specific restorer. For example, the data from DatabaseDumper is transferred to DatabaseRestorer. Each restorer reproduces a piece of your system and the request is made when the entire system is in the same state as in the staging server.

Database Restorer

In order to illustrate how the restorers work, let me explain precisely the database restore process. The first thing is receiving the data from CrashBreak. As I mentioned before, DatabaseDumper sends nothing to the website but the dump’s name. The restorer takes this name and uses it to download the dump file from your server. Here’s an example of AWS settings:

config.dumper_options = {

aws_bucket_name: 'my-test-app',

aws_region: 'us-east-1', # default: ENV['AWS_REGION']

aws_access_key_id: '', # default: ENV['AWS_ACCESS_KEY_ID']

aws_secret_access_key: '' # default: ENV['AWS_SECRET_ACCESS_KEY']

}

When the dump is already on your local computer, the restorer creates an empty database and reproduces the dump into it. In the next step, a connection to the newly-created database is created. That’s all. You’re ready to send a request.

External Integration

There are several integrations that can make your life easier and increase the automation of the process.

GitHub

With the GitHub integration, a new branch with a test is automatically created as soon as an exception appears. Your work is reduced to downloading the branch, fixing the bug and pushing the fix to GitHub.

CI server

A request test of CrashBreak can be also run on a CI machine. Think about integration with GitHub and CI together. You can send the fix, the CI machine will check it for you. If the commit fixes the error, the CI machine will create a pull request on GitHub, and will mark the error as resolved in the CrashBreak system.

A Reward For The Long Setup

After all setup, the fear of deploying server on Friday should be gone and you are prepared for exceptions. Also the flow of fixing bugs is more comfortable for programmers, with the use of all integrations, it’s possible to reduce your work to:

- downloading a branch with an error

- fixing the error and sending back the code

- merging the pull request

Sounds better than reproducing each error manually in the browser? But there’s one more thing…

Does It Really Work?

The shortest answer is:

It’s not perfect, though. There’s still a lot to be done. Many dependencies don’t have a proper dumper. For example, external API requests could be saved and sent in the dumper, and then mocked in the restorer, so that they can return the same results as in the staging server.

Extensions

CrashBreak is made in a very adaptive way. You can write your own dumpers, restorers, serializers and adapt the test file to your needs.

Links

GitHub: https://github.com/crashbreak

CrashBreak site: http://crashbreak.com/

Extensions: http://www.crashbreak.com/extensions/

Thank you!

What do you think about my idea? I’ll be delighted to hear your comments. You can contact me via email contact@crashbreak.com or just write a comment here!