I like to write unit tests for my code and I also like to know whether my unit tests are actually testing my code. In Java I would use Maven and Cobertura to measure how

much of my code is actually exercised, and when I bend my mind around Haskell I use HPC.

I have been experimenting with Clojure and it has a nice testing library but the only way I have found to measure coverage is to use Cobertura and squint at the generated class names to work out what I haven’t covered. So I have started on the road to creating a coverage tool for Clojure written in Clojure. This blog entry contains my initial steps and design sketches towards creating the tool.

Ideally I’d like to measure my coverage like this:

( ns some . tests

( : use [ some . functions ] )

( : use [ coverage . cover ] )

( : use [ clojure . test ] ) )



( deftest test - something

( cover ' ( foo bar ) ; cover foo and bar

( is ( = [ 1 2 3 ] ( foo 1 2 3 ) ) )

( is ( = [ 2 4 6 ] ( bar 1 2 3 ) ) )



( is ( = [ 100 100 ] ( coverage foo ) ) ) ; check coverage

( is ( = [ 100 100 ] ( coverage bar ) ) )



( store - coverage ) ) ) ; store report to file

That isn’t functional code yet and may not be what I finally produce, it might not even be valid Clojure since I haven’t even bashed it into a REPL yet.

As Clojure is a Lisp, and we are testing our own code – i.e. we have the source code – we should be able to measure the coverage using Clojure. This can achieved by rebinding the functions under test with instrumented functions;

this is what I envisage the cover function is doing:

( cover ' ( foo )

; all calls to foo here actually call foo'

)

Which can be achieved using a binding function:

( binding [ a b ) ]

; all calls to a here are actually calls to b

)

So assuming I can write the cover function I will need to instrument a function. Let’s start simply and ignore the complicated stuff such as conditional statements and multiple bodies until later. If we have this simple function

( defn foo [ a b c ] ( + a ( - b c ) ) )

I would like an instrumented version to wrap each s-expression with a function that records that the s-expression has been called, for now we will just print a message.

( defn foo - wrapped [ a b c ]

( do

( prn : wrap )

( + a

( do

( prn : wrap )

( - b c ) ) ) ) )

Testing that in the REPL seems to reveal that I am moving in the right direction:

user => ( foo 1 2 3 )

0

user => ( foo - wrapped 1 2 3 )

: wrap

: wrap

0

user =>

To convert the first function to the instrumented function I need a couple of auxiliary functions and a macro:

( defn record [ ] ( prn : wrap ) )



( declare wrap - seq )

( defn wrap [ form ]

( cond

( seq? form )

( list ' do ( record ) ( wrap - seq form ) )

: else form ) )



( defn wrap - seq [ xs ]

( for [ x xs ] ( wrap x ) ) )



( defmacro cover

[ args & amp ; body]

( let [ fn - name ( when ( symbol? args ) args )

args ( if fn - name ( first body ) args )

body ( if fn - name ( next body ) body ) ]

( fn ~@ ( if fn - name ( list fn - name args ) ( list args ) )

~@ ( wrap ( first body ) ) ) ) )

If you know some Clojure that code demonstrates some of the problems I haven’t solved yet when measuring coverage, for example the :else clause will need to be covered as well as functions in binding statements. Testing that code in the REPL gives us a reasonable result:

user => ( def x ( cover [ a b c d ] ( + a ( - b c ( * a d ) ) ) ) )

#'user / x

user => ( x 1 2 3 4 )

: wrap

: wrap

: wrap

- 4

user =>

So if I now replace the record function with something else I may be able to generate some coverage for a function. I am going to use a map held by a ref to store the coverage information, a function to register a particular

s-expression with the ref and a function to increment the count.

( def coverage - records ( ref nil ) )



( defn covering [ form ]

( dosync ( alter coverage - records assoc form 0 ) ) )



( defn- inc - map [ map key ]

( if ( contains? map key )

( assoc ( dissoc map key ) key ( inc ( get map key ) ) )

map ) )



( defn inc - coverage [ key ]

( dosync ( alter coverage - records inc - map key ) ) )

Lets test that:

user => @coverage - records

nil

user => ( covering : x )

{ : x 0 }

user => ( covering : y )

{ : y 0 , : x 0 }

user => ( inc - coverage : x )

{ : x 1 , : y 0 }

user => ( inc - coverage : y )

{ : y 1 , : x 1 }

user => ( inc - coverage : x )

{ : x 2 , : y 1 }

user => @coverage - records

{ : x 2 , : y 1 }

user =>

If I modify our wrapping functions I can create the coverage records ready to be incremented.

( defn wrap [ form ]

( cond

( seq? form ) ( do

( covering ( str form ) )

( list ' do ` ( record ) ( wrap - seq form ) ) )

: else form ) )

Testing that code:

user => ( def x ( cover [ a b c d ] ( + a ( - b c ( * a d ) ) ) ) )

#'user / x

user => @coverage - records

{ "(* a d)" 0 , "(- b c (* a d))" 0 , "(+ a (- b c (* a d)))" 0 }

user =>

And that is as far as I have got. I need to change my record function to increment the coverage records, make my wrap function cope with conditionals and other complexities and write my rebinding function. Hopefully next month I will have moved it along a little bit further.