This article is part of a series describing a port of the samples from Practical Common Lisp (PCL) to Clojure. You will probably want to read the intro first.

This article covers Chapter 9, Practical: Building a Unit Test Framework.

Tests and reports

To build a minimal testing library, I need nothing more than tests and results. To keep reporting as simple as possible, I will start with console output. The report-result function tests a result , and prints pass or FAIL , plus a form with supporting detail:

( defn report-result [ result form ] ( println ( format "%s: %s" ( if result "pass" "FAIL" ) ( pr-str form ))))

Now any function can be a test. The detail message can often be the same form that caused the error, so I will pass the same form twice: once for evaluation, and again (quoted!) for use in the detail message:

( defn test-+ [] ( report-result ( = ( + 1 2 ) 3 ) ' ( = ( + 1 2 ) 3 )) ( report-result ( = ( + 1 2 3 ) 6 ) ' ( = ( + 1 2 3 ) 6 )) ( report-result ( = ( + -1 -3 ) -4 ) ' ( = ( + -1 -3 ) -4 )))

The console output for test-+ looks like this:

user=> ( test-+ ) pass : ( = ( + 1 2 ) 3 ) FAIL : ( = ( + 1 2 3 ) 7 ) pass : ( = ( + -1 -3 ) -4 )

Inferring the detail message

The fact that I want to pass the same form twice, but with different evaluation semantics, just screams macro. Sure enough, I can clean up the code with a macro:

( defmacro check [ form ] ` ( report-result ~ form '~ form ))

The macro expands the form twice, once for evaluation and once quoted for the detail message. Now I can replace calls to report-result with simpler calls to check :

( defn test-* [] ( check ( = ( * 1 2 ) 3 )) ( check ( = ( * 1 2 3 ) 6 )) ( check ( = ( * -1 -3 ) -4 )))

Hmm. The calls to check are cleaner than the calls to report-result in the earlier example, but the check itself still looks repetitive. Solution: a better check macro that can handle multiple forms:

( defmacro check [ & forms ] ` ( do ~@ ( map ( fn [ f ] ` ( report-result ~ f '~ f )) forms )))

The quoting and unquoting is a little more complex–play around with macroexpand-1 to see how it works.

With the better check in place, test functions are quite simple:

( defn test-rem [] ( check ( = ( rem 10 3 ) 1 ) ( = ( rem 6 2 ) 0 ) ( = ( rem 7 4 ) 3 )))

Aggregating results

So far I have tests and console output. Next, I need some way to aggregate a set of checks into a single, top-level "checks passed" or "checks failed".

I would like to simply and together all the individual checks, but that does not quite work. As in many languages, Clojure's and short-circuits and stops evaluating when it encounters a logical false . That's no good here: Even if one test fails, I still want all the tests to run.

Since it is a question of optional evaluation, a macro is appropriate. The combine-results macro works like and , but it always evaluates all the forms:

( defmacro combine-results [ & forms ] ` ( every? identity ( list ~@ forms )))

Now check can use combine-results instead of do .

( defmacro check [ & forms ] ` ( combine-results ~@ ( map ( fn [ f ] ` ( report-result ~ f '~ f )) forms )))

All existing functionality still works, and now I can see a useful return value from a test.

user=> ( test-* ) pass : ( = ( * 2 4 ) 8 ) pass : ( = ( * 3 3 ) 9 ) true

Capturing test names

Tests ought to have names. In fact, tests ought to support multiple names. You can imagine a test detail report saying:

Check math->addition->associative passed: ...

Where associative is the name of a check, addition is the name of a function, and math is the name of another function that called addition .

First, I need a variable to store a sequence of names:

( def *test-name* [])

Printing the variable as part of a result is easy:

( defn report-result [ result form ] ( println ( format "%s: %s %s" ( if result "pass" "fail" ) ( pr-str *test-name* ) ( pr-str form ))) result )

Now for the hard part: populating the collection of names. For this, I will introduce a deftest macro:

( defmacro deftest [ name & forms ] ` ( defn ~ name [] ( binding [ *test-name* ( conj *test-name* ( str '~ name ))] ~@ forms )))

The macro expansion perfomed by deftest is nothing new: deftest turns around and defn s a new function named name . The interesting part is the call to binding , which rebinds *test-name* to a new collection built from the old *test-name* plus the name of the current test.

The new binding of *test-name* is visible anywhere inside the dynamic scope of the binding form. The dynamic scope includes any function calls made inside the binding, and their function calls, and so on ad infinitum … or until another binding performs the same trick again. This gives exactly the semantics we want:

The dynamic scope allows callers to influence callees without having to pass test-name an an argument all over the place. Nested functions "remember" a stack of their caller's names through *test-name* .

an an argument all over the place. Nested functions "remember" a stack of their caller's names through . The unwinding of the dynamic scope protects readers of *test-name* outside a binding . Code after the binding will never see the values *test-name* takes during the binding .

outside a . Code after the will never see the values takes during the . Dynamic bindings are thread-local (and therefore thread-safe).

With deftest in place, I can defined a hierarchy of nested tests:

( deftest test-* ( check ( = ( * 2 4 ) 8 ) ( = ( * 3 3 ) 9 ))) ( deftest test-math ; TODO: test rest of math ( test-* )) ( deftest test-all-of-nature ; TODO: test rest of nature ( test-math ))

Calling test-all-of-nature will demonstrate multiple levels of nested name in a test report:

user=> ( test-all-of-nature ) pass : [ "test-all-of-nature" "test-math" "test-*" ] ( = ( * 2 4 ) 8 ) pass : [ "test-all-of-nature" "test-math" "test-*" ] ( = ( * 3 3 ) 9 ) true

From here, better formatting of the console message is just mopping up.

Wrapping up

When I first read Practical Common Lisp, this was my favorite chapter. The testing library evolves quickly and naturally to a substantial feature set. (In case you didn't keep count, the entire "framework" is less than twenty lines of code.)

Try implementing the unit-testing example in your language of choice. Don't just implement the finished design. Work through each of the iterations described above:

tests and results inferring the detail message aggregating results capturing test names

I would love to hear about your results, and I will link to them here.

Notes