Back in September 2008, I had a list of the most popular testing modules on the CPAN. I created this list because I was writing Test::Most and I needed to know them. Tonight, after hearing that the next Perl-QA hackathon is in Vienna, I thought about what I might want to accomplish and decided to see if the most popular modules had changes. In 2008, out of 373 test modules, we had the following top 20:

1 Test::More 69396 2 Test 10912 3 Test::Exception 2314 4 Test::Simple 962 5 Test::Base 610 6 Test::NoWarnings 386 7 Test::Builder::Tester 308 8 Test::Deep 254 9 Test::Pod 223 10 Test::Warn 217 11 Test::Differences 213 12 Test::MockObject 187 13 Test::Pod::Coverage 157 14 Test::Builder 131 15 Test::WWW::Mechanize::Catalyst 118 16 Test::XML 113 17 Test::Block 112 18 Test::Perl::Critic 110 19 Test::Distribution 107 20 Test::SQL::Translator 101

In January 2010, out of 493(!) Test:: modules, we have the following top 20:

1 Test::More 86194 2 Test 10789 3 Test::Exception 3548 4 Test::Base 1006 5 Test::Simple 962 6 Test::NoWarnings 884 7 Test::Deep 413 8 Test::Warn 350 9 Test::Pod 335 10 Test::Builder::Tester 322 11 Test::Differences 311 12 Test::Pod::Coverage 277 13 Test::Perl::Critic 256 14 Test::MockObject 205 15 Test::Builder 203 16 Test::Most 170 17 Test::Block 169 18 Test::WWW::Mechanize::Catalyst 162 19 Test::Distribution 150 20 Test::Kwalitee 143

That's when I noticed something very interesting. My Test::Most module, the one I created this list for, was already in the number 16 slot (it's currently at the number 13 slot if you ignore modules in the Test:: namespace but which don't export test functions)! That's when my failure dawned on me. Given that people who use Test::Most are more serious about testing, it's reasonable that they're more serious about testing, thus artificially inflating my stats. I was counting the number of times the modules were being used. However, if I write, say, an extra 160 test programs for my modules, each of which uses Test::Most (or indeed, just type use Test::Most 160 times in a single module), then I could push my module into the top ten. So I reran my stats per distribution:

1 Test::More 14111 2 Test 1736 3 Test::Exception 744 4 Test::Simple 331 5 Test::Pod 328 6 Test::Pod::Coverage 274 7 Test::Perl::Critic 248 8 Test::Base 228 9 Test::NoWarnings 155 10 Test::Distribution 142 11 Test::Kwalitee 138 12 Test::Deep 128 13 Test::Warn 127 14 Test::Differences 102 15 Test::Spelling 101 16 Test::MockObject 87 17 Test::Builder::Tester 84 18 Test::WWW::Mechanize::Catalyst 79 19 Test::UseAllModules 63 20 Test::YAML::Meta 61 21 Test::Synopsis 57 22 Test::Compile 56 23 Test::Portability::Files 54 24 Test::Most 49

OK, so I dropped down to a more reasonable 24th place. But a quick check of CPANTS showed that I wasn't finding as many uses of Test::Most as I should because there were cases where:

Authors declared it in their Makefile.PL or Build.PL but didn't use it

or but didn't use it In at least one case (probably more), the author has tests in their lib/ directory that I was not searching!

There was also the case where it turns out that Robert Krimen is a huge fan of Test::Most (Hi Robert!) and has enough modules out there to artificially inflate my stats. So here's yet another list, showing test modules per author:

1 Test::More 2669 2 Test 685 3 Test::Exception 260 4 Test::Simple 161 5 Test::Pod 76 6 Test::Warn 66 7 Test::Perl::Critic 65 8 Test::Deep 60 9 Test::Pod::Coverage 55 10 Test::NoWarnings 51 11 Test::Differences 50 12 Test::Builder::Tester 49 13 Test::MockObject 49 14 Test::Base 47 15 Test::WWW::Mechanize::Catalyst 36 16 Test::Builder 27 17 Test::Kwalitee 26 18 Test::MockObject::Extends 24 19 Test::Distribution 21 20 Test::Output 19 21 Test::XML 17 22 Test::Moose 16 23 Test::Harness 16 24 Test::LongString 15 25 Test::Tester 13 26 Test::Spelling 12 27 Test::MockModule 11 28 Test::UseAllModules 10 29 Test::TCP 10 30 Test::Most 10

That seems a touch more reasonable, though I confess I'd like to see more folks using this module. It really does make your testing life simpler.

And as for the beautiful Test::Class which more people should be using:

95 test programs (25th place)

19 distributions (42nd place)

9 authors (33rd place)

Come on, folks! You can do better than that, right?