Creating custom Minitest reporters By Nando Vieira April 30, 2015 Read in 4 minutes

Minitest it’s having some momentum right now. I’ve been using it on some personal projects and, coming from RSpec, I really missed its awesome execution report. Minitest has some 3rd-party reporters, but they don’t get any close to RSpec’s awesomeness.

So I decided to dive into Minitest source code and learn how to create a custom Minitest reporter similar to RSpec’s reporter. Here’s the final result:

Note that it even gives you the command for running only the test that failed, pretty much like RSpec. Sweet! By the way, if you want to have this reporter too, just install minitest-utils.

It turns out creating custom reporters in Minitest is a really simple task, so I’ll guide you in this article, creating two different reporters.

Understanding Minitest’s plugin system

Minitest’s plugin system relies on file discovery. It will load all files present on your $LOAD_PATH within the minitest/*_plugin.rb pattern. Let’s say I’m creating a gem called minitest-utils . I can create a file at lib/minitest/utils_plugin.rb and Minitest will load it. This file can implement two methods on Minitest module, as you can see below.

module Minitest def self . plugin_utils_init ( options ) # ... end def self . plugin_utils_options ( opts , options ) # ... end end

Your methods have to follow the Minitest.plugin_*_init and Minitest.plugin_*_options naming scheme. Let’s breakdown these methods’ responsibility.

First, if you intend to extend the CLI, you have to implement the Minitest.plugin_*_options method. It receives two parameters containing the OptionParser instance and a hash containing options like the output IO instance. One could extend the CLI including options for displaying colored test output.

module Minitest def self . plugin_utils_options ( opts , options ) opts . on ( '--color' , 'Display colored output' ) do | color | options [ :color ] = color end end end

Minitest’s default reporter accepts multiple reporters, implemented as the Minitest::CompositeReporter class. For custom reporters, and other extensions, you have to implement the Minitest.plugin_*_init method, adding your reporter to the composite reporter.

module Minitest def self . plugin_utils_init ( options ) Minitest . reporter << GrowlReporter . new ( options [ :io ], options ) end end

Now that you know how the plugin system works, let’s create a custom reporter.

Creating a custom reporter

Here’s how Minitest’s default reporter looks like.

Not that useful. So, let’s make it shine with colors.

For this to work, we’ll need to clear out default reporters from the composite reporter. We can do it in the Minitest.plugin_utils_init method.

module Minitest def self . plugin_colored_reporter_init ( options ) Minitest . reporter . reporters . clear Minitest . reporter << ColoredReporter . new ( options [ :io ], options ) end end

As you can see, we’re using a ColoredReporter class. Let’s define it.

class ColoredReporter < Minitest :: StatisticsReporter end

Now, you have to define the methods used for reporting. The first one is ColoredReporter#record , which is called after every test execution. This is the place we’ll output the realtime progress with all those dots and other failing indications.

class ColoredReporter < Minitest :: StatisticsReporter def record ( result ) super result_code = result . result_code io . print color ( result_code , RESULT_CODE_TO_COLOR [ result_code ]) end end

Note that we’re calling super ; we have to do this so that Minitest::StatisticsReporter can compute the result status counting. Now, we have to implement the color method, which is pretty straight-forward.

class ColoredReporter < Minitest :: StatisticsReporter RESULT_CODE_TO_COLOR = { 'S' => :yellow , '.' => :green , 'F' => :red , 'E' => :red } COLOR_CODE = { red: 31 , green: 32 , yellow: 33 , blue: 34 , none: 0 } def record ( result ) super result_code = result . result_code io . print color ( result_code , RESULT_CODE_TO_COLOR [ result_code ]) end def color ( text , color = :none ) code = COLOR_CODE [ color ] " \e [ #{ code } m #{ text } \e [0m" end end

Next, we have to implement the ColoredReporter#report method. This is where we show which tests failed, and can also display the statistics of the executed tests. In this case, we’ll basically call some methods.

class ColoredReporter < Minitest :: StatisticsReporter # ... def report super io . puts "



" io . puts statistics io . puts aggregated_results io . puts summary end end

The ColoredReporter#statistics method will calculate the execution time, and some other info.

class ColoredReporter < Minitest :: StatisticsReporter # ... def statistics "Finished in %.6fs, %.4f runs/s, %.4f assertions/s." % [ total_time , count / total_time , assertions / total_time ] end end

The ColoredReporter#aggregated_results method will display the tests that failed or were skipped.

class ColoredReporter < Minitest :: StatisticsReporter # ... def aggregated_results filtered_results = results . sort_by { | result | result . skipped? ? 1 : 0 } filtered_results . each_with_index . map { | result , i | color ( "

%3d) %s" % [ i + 1 , result ], result . skipped? ? :yellow : :red ) }. join + "

" end end

Finally, let’s display the test execution summary, with number of tests, assertions, failures, and more.

class ColoredReporter < Minitest :: StatisticsReporter # ... def summary summary = "%d runs, %d assertions, %d failures, %d errors, %d skips" % [ count , assertions , failures , errors , skips ] color = :green color = :yellow if skips > 0 color = :red if errors > 0 || failures > 0 color ( summary , color ) end end

And we’re done! Now we have more visibility about what’s happening in our tests, as you can see below.

One more example

What about displaying Growl notifications (or any other notification system you may have)? Using the test_notifier gem, this is quite simple.

class TestNotifierReporter < Minitest :: StatisticsReporter def report super stats = TestNotifier :: Stats . new ( :minitest , { count: count , assertions: assertions , failures: failures , errors: errors }) TestNotifier . notify ( status: stats . status , message: stats . message ) end end

You can add it to the composite reporter just like before.

module Minitest def self . plugin_colored_reporter_init ( options ) Minitest . reporter . reporters . clear Minitest . reporter << ColoredReporter . new ( options [ :io ], options ) Minitest . reporter << TestNotifierReporter . new end end

On Mac and Growl, this is what you’ll have:

Wrapping up

Minitest is really lightweight and extensible. I never really cared about the assertion syntax (expect vs should vs assert vs must/wont), but RSpec’s feedback made all the difference. It gives you the command for running failing tests and allows you to rapidly view what went wrong. Now that I have the same level of feedback, I’ll probably stick to it. Rails has so many custom test case classes (mailers, jobs, generators, and more) that I can just use it, instead of trying to figure out how to do it with RSpec.