Porting your test suite to MiniTest, part 2: how to port your tests

This is the last part of a series I wrote about porting Noko’s test suite to MiniTest. Missed the first part?

About a month ago we ported Noko’s test suite from test-spec to MiniTest. Doing so not only made our test suite faster, but it also paved the road for upgrading Rails in the future. Plus, we now have Nyancat running our tests

While porting over a test suite is not a technically challenging problem, it is certainly daunting. You’re changing thousands of lines of code, and sometimes you have to rethink tests entirely. And to make matters worse, your tests don’t just go from “green” to “red”. Instead, they go from “green” to “OH GOD, WHY?! NOTHING WORKS, EVERYTHING IS ON FIRE!”, which can be pretty demoralizing. But thankfully with some patience, a few regular expressions, and by divying up the work; the process becomes relatively painless.

Previously, I talked about some of the benefits of porting your tests to MiniTest and using the “spec-assert” style. Now I’m going to talk about the nitty-gritty of actually getting the test suite ported. First up are some general workflow tips to keep in mind when porting. After that I’ll talk about some of the technical details and solutions to “gotchas” you’re bound to run into. Finally, I’ve included some regular expressions that make porting a breeze. Just plug them into your text editor, and find-and-replace away!

How to port your test suite

Use the “Ruby on Rails Testing Cheat Sheet”

Seriously, this cheat sheet (written by my friend Eric Steele) saved me hours of time: https://whatdoitest.com/rails-assertions-cheat-sheet. You should buy his book, What do I test?, because it’s fantastic and has the best DHH testimonial ever:

I appreciate the honesty on the part of @dhh and @genericsteele regarding Eric's book. That's my kind of endorsement: pic.twitter.com/1HHVW6r1Mp — Ben Halpern (@bhalp1) August 7, 2014

Only work on one type of test at a time

Functional tests are different than unit tests, which are different than integration tests, and so on. By only focusing on one type of test at a time it’s easier to get in a groove and crank them out. And splitting the team up to work on different types of tests makes the porting faster and makes sure you don’t step on each other’s toes.

Only work on one file at a time

The quickest way to be overwhelmed and not make any progress is to run:

rake test

Instead, work on files in isolation:

ruby test/unit/user_test.rb

When all the tests in a file pass, move onto the next one. You’ll probably run into errors caused by running all the tests from rake test , but wait until you’ve fixed them individually before you tackle that.

Pick some good music

Porting these tests can take a while and it can be pretty monotonous, so you’ll want to find something that will stop you from chanting “Redrum”. I ended up listening to the music from Cowboy Bebop when I was porting tests, and it made the process enjoyable.

Regular expressions all the way down

Seriously, the best way port a test suite is to heavily use regular expressions and find-and-replace. Your tests should have a (relatively) standard format, which makes them prime targets for some well-crafted regexes. Here are some regular expressions to get your started.

Technical details

We ran into a few technical hiccups when porting tests from test-spec and MiniTest. Thankfully they were all pretty easy to solve.

Setting up MiniTest

To set up your test suite to use MiniTest, you’ll have to update your Gemfile and test_helper.rb .

Here are the gems you’ll probably need to install:

minitest-spec-rails : this forces ActiveSupport::TestCase to use the MiniTest::Spec::DSL . Works with Rails 2.3, 3.x, 4.x

: this forces to use the . Works with Rails 2.3, 3.x, 4.x minitest-spec-rails-tu-shim : makes Test::Unit compatible for MiniTest in Ruby 1.8, supports minitest-spec-rails

You’ll also have to update test_helper.rb to include the following:

require 'action_view/test_case' require 'minitest-spec-rails' require 'test_help' require 'webmock/minitest' #if you're using webmock require 'test_helper/add_allow_switch' #if you used test-specs `add_allow_switch` require 'test_helper/deprecated_helpers' #if you use the deprecated helpers gist below require 'test_helper/fix_nested_describes_in_controller_tests' #for Rails 2.3 LTS only require 'test_helper/test_assertions' #adds a useful assertions

If you were using test-spec ’s add_allow_switch helper, you’ll want to add the following gist to your test helpers: https://gist.github.com/tcannonfodder/8b3fb3ba9e838a436a2f

You can add the following test helper, which will provide helpful deprecation warnings for some test-spec helpers: https://gist.github.com/tcannonfodder/96fa736ec5c67cf43d1c.

The following test helper fixes a bug with nested describes blocks in functional tests for Rails 2.3: https://gist.github.com/tcannonfodder/82a7c7216506b4b20859

The following test helper adds a lot of helpful asserts to your test suite, like ordered array and set comparison ( assert_equal_list and assert_equal_set respectively), ActiveRecord attribute validation, and layout assertions: https://gist.github.com/tcannonfodder/2b35449aacd76dbd1c54

Wrap your tests in a class, even if you’re using a describe block

minitest-spec-rails doesn’t properly recognize non-model classes, which will cause random errors in your test suite. Thankfully, you can nest describe blocks inside of class declarations, so just wrapping each test file with the appropriate class declaration ensures nothing breaks:

# Unit tests for models class UserTests < ActiveSupport :: TestCase describe User , "permissions" do it "should have permissions to view" do assert user . has_permission? ( :view ) end end end # Functional tests (app/controllers) class UsersControllerTest < ActionController :: TestCase describe UsersController , "#show" do it "should a user" do get :show , :id => users ( :bob ). id assert_equal_set users ( :bob ), assigns ( :user ) end end end # Integration Test class UserIntegrationTest < ActionController :: IntegrationTest describe "Getting Users" do it "should get a user" do get "/users/ #{ users ( :bob ). id } " assert_select '#name' , :text => "Bob Kerbin" end end end # Unit test for non-model class (e.g: `lib/`) class NameParsingTest < ActiveSupport :: TestCase describe NameParsing do it "should be able to intelligently parse a first name" do assert_equal "Bob" , NameParsing . parse_name ( "Bob Kerbin" ). first_name end end end

before and after blocks execute for every test

There are no longer different before/after :each or before/after :all blocks. You only have before/after do ... end , which runs for every test in the scope.

Performance testing in Rails 2.3 LTS is broken without a monkeypatch

Rails 2.3’s performance testing framework doesn’t like MiniTest when is used instead of Test::Unit. This problem was fixed in Rails 3+, and I’ve written a monkey-patch that will add MiniTest support to ActiveSupport::Testing::Performance : https://gist.github.com/tcannonfodder/e4ea2d7f877bdcc5dcdf

Brush up on your Regex-fu

The biggest part of porting from “spec-shoulda” to “spec-assert” was rewriting the assertions in a test. Thankfully these should be fairly standard, which means you can use regular expressions to quickly knock out tests. And since the rest of your team is also porting tests, sharing the regular expresisons you’ve created will make everyone’s job easier

Tip: go from most → least specific regular expression

Sometimes the general-case regular expressions will gobble up a test that would be better expressed with a different assertion. For example, consider the following test:

user . valid? . should == true

This test should be rewritten as:

assert user . valid?

But when you use the x.should == y replacement regexes, you’ll end up overwriting the test to:

assert_equal true , user . valid?

If you modify the regex to check for x.should == true and replace it with assert x , you’ll keep your tests nice and clean.

Here are some few regular expressions to get your started

x.should include y Find : ([^\S

]+)(.*)\.should\.include\s+(.*) Replace : $1assert_include $2, $3

response.status.should be x Find : ([^\S

]+)response\.status\.should\.be\s+(.*) Replace : $1assert_response $2

should.redirect_to x Find : should\.redirect_to\s*(\w+) Replace : assert_redirected_to $1

x.should.be.empty Find : ([^\S

]+)(.*)\.should\.be\.empty Replace : $1assert_empty $2

x.should.be.blank Find : ([^\S

]+)(.*)\.should\.be\.blank Replace : $1assert_blank $2

x.should.be.y Find : ([^\S

]+)(.*)\.should\.be\.+(.*) Replace : $1assert $2.$3?

x.should.not.be.y Find : ([^\S

]+)(.*)\.should\.not\.be\.+(.*) Replace : $1assert !$2.$3?

lambda{ do_stuff }.should.raise(x) Find : lambda\s*\{([^}]*)(\s)*\}\.should\.raise\((.*)\) Replace : assert_raises $3 do$1end

lambda{ do_stuff }.should.not.raise(x) Find : lambda\s*\{([^}]*)(\s)*\}\.should\.not\.raise\((.*)\) Replace : assert_nothing_raised do$1end

lambda{ do_stuff }.should.change(x) Find : lambda\s*\{([^}]*)\}\.should\.change\((.*)\) Replace : assert_difference $2 do$1end

lambda{ do_stuff }.should.not.change(x) Find : lambda\s*\{([^}]*)\}\.should\.not\.change\((.*)\) Replace : assert_no_difference $2 do$1end

x.should.be.nil Find : ([^\S

]+)(.*)\.should\.be\.nil Replace : $1assert_nil $2

x.should.not.be.nil Find : ([^\S

]+)(.*)\.should\.not\.be\.nil Replace : $1assert_not_nil $2

lambda do some_expression end.should.raise(x) Find : lambda\s*do([^}]*?)end.should.raise\((.*)\) Replace : assert_raises $2 do$1end not perfect, will break if you have a } in the lambda

x.should.message("does not equal") == y Find : ([^\S

]+)(.*)\.should\.messaging\(\s*"(.*)"\s*\)\s*==\s*(.*) Replace : $1assert_equal $4, $2, "$3"

x.should =~ y Find : ([^\S

]+)(.*)\.should\s+=~\s+(.*) Replace : $1assert_match $3, $2

x.select{ |y| some_expression }.should.not.be.empty Find : ([^\S

]+)(.*)\{([^}]*)\}\.should\.not\.be\.empty Replace : $1assert_not_empty $2{$3}

x.should == true Find : ([^\S

]+)(.*)\.should\s+==\s+true Replace : $1assert $2

x.should != true Find : ([^\S

]+)(.*)\.should\s+!=\s+true Replace : $1assert !$2

x.should == false Find : ([^\S

]+)(.*)\.should\s+==\s+false Replace : $1assert !$2

x.should == y Find : ([^\S

]+)(.*)\.should\s+==\s+(.*) Replace : $1assert_equal $3, $2

x.should.equal y Find : ([^\S

]+)(.*)\.should\.equal\s+(.*) Replace : $1assert_equal $3, $2

x.should.not.equal y Find : ([^\S

]+)(.*)\.should\.not\.equal+(.*) Replace : $1assert_not_equal $3, $2

x.should.not == y Find : ([^\S

]+)(.*)\.should\.not\s+==\s+(.*) Replace : $1assert_not_equal $3, $2

x.should != y Find : ([^\S

]+)(.*)\.should\s+!=\s+(.*) Replace : $1assert_not_equal $3, $2



That’s it!

Porting your test suite to MiniTest and rewriting from “spec-shoulda” to “spec-assert” styles can be a long process, but it has some significant benefits. It makes your test suite run faster by reducing overhead. It also makes your tests easier to scan and understand, which is critical during code reviews and introducing new members to the team. Finally, it gets you closer to Rails’ defaults, which makes it easier to upgrade when the time comes.