I had used Mocha in the past and recalled liking it. It also claimed to support generating tests dynamically, which is something I couldn’t find in the Jest documentation. However, I wanted to write dynamic tests based on data that was fetched asynchronously. This simple test case demonstrates that generating tests asynchronously doesn’t work very well:

Mocha fails to generate dynamic tests asynchronously

If you try and run it, Mocha will report that there are no tests. There are workarounds, like the one detailed in this SO answer, but they aren’t very clean. I wanted something better.

After scouring the internet I found this article that I really liked. I agreed that abuse of globals and shared state were big problems with Mocha, Jasmine, Jest, and the others. I especially liked the quote “Test assertions should be dead simple & completely free of magic”. The article also introduced me to the Test Anything Protocol (TAP) which I was previously unfamiliar with. The article argued that you should use the Tape package. Reading through the documentation on Tape, the first thing that I noticed was that if I wanted support for Promises, I needed to use a spinoff called Blue Tape. The Blue Tape README has some example pseudo code so I modified it so that it would run and then tested it out. As you can see here, the results were less than thrilling. This was code from their one example and it didn’t even run correctly. That told me everything I needed to know about the library.

At this point I was about to resign myself to using Mocha. But a little more searching and I stumbled across Node-Tap. I found the Node-Tap documentation directly, rather than finding a recommendation for it. This made it seem like an unused package that someone had built as a weekend project and then abandoned at first glance. The more I read though, the more excited I got. This was finally the package I was looking for! The author lays out several of his opinions on testing libraries most of which I thought were very insightful. The first one was profound enough that it bears repeating here:

Test files should be “normal” programs that can be run directly.

This really struck a chord with me. It means that your tests shouldn’t have global variables that new developers puzzle about in wonder. It means that if you want to add test cases using a for loop, you can. It means that if you want to put test cases in a setTimeout callback, you can. If you are writing tests at work than that means you write code for a living. If you write code for a living than you can probably handle writing code for tests.

Eagerly, I installed node-tap and wrote my first test cases. But as the node-tap author pointed out “tutti i gusti, sono gusti” (there is no accounting for taste). The package upheld its promises, but there was still something I really didn’t like about it. Take a look at the following code:

Can you spot the problem? For those of you who don’t see it, the problem is that plan or end is never called and so the test will fail because it is “unfinished”. If you did spot the problem then you’ve probably been indoctrinated by other test frameworks. Imagine for a second that you are a seasoned developer, but that you’ve never seen tests before in your life. Looking at the code you can see that the function tap.test is passed a string and a function, which is probably some sort of callback. You look at the callback and can recognize that the assert will not throw an error. So you reason that if the assert were to throw an error, the test would probably fail. Likewise, you reason that since the assert won’t throw then the test will probably pass. But that isn’t the case. Instead we have a framework that will behave counterintuitively to the average person.

I can already hear people crying out in protest: but plan helps me avoid false positives! Take a look at the following two test cases:

Test cases the do and don’t need a plan

The first test case needs a plan . If plan is not required and someCondition and anotherCondition are both false then the test will pass even though you may have expected it to fail anytime anotherCondition is false. But if you refactored it to look like the second test then you don’t have this problem. In addition, there should probably be two test cases, one where you assert that someCondition is true and one where you assert that it is false. As an open challenge, I ask you to think of an example where the test case can’t be refactored so as to eliminate the need for plan .

At this point I was thoroughly discouraged. But, as I argued earlier, if you can write code, than you can write code for tests. And if you can write code for tests, than you can write code for a test runner. So I took it upon myself to write a test runner for Javascript. The result was cupping.

It is a small, simple library that tries to exemplify the principles that I thought were important while avoiding the aforementioned pitfalls of other testing libraries. Go ahead and install it, try it out, and let me know what you think. It might not be for everybody, but I hope that there are at least some people out there who will be delightfully refreshed when they use it.