Clojure micro benchmark

September 29, 2010 at 7:15 pm | Posted in Clojure, Programming | 6 Comments



First a disclaimer: benchmarking software is difficult. Furthermore, micro-benchmarks hardly ever make sense, unless you have identified that particular piece of code as a performance bottleneck. Following Donald Knuth’s famous quote: “premature optimization is the root of all evil”. Having said this, I also know that tinkering with a few lines of code and trying different versions can be a lot of fun. So that is exactly what I did when I needed some Clojure code that calculates the sum of squares of two sequences.

The algorithm itself is straightforward: it iterates through all the values of the two sequences, calculates the square of the difference for each value pair, and sums them. A perfect algorithm for a map/reduce implementation.

The first version sumsqr1 uses an anonymous function (#(* % %)) to calculate the square root. The advantage is that the subtraction y from x is only done once. The downside might be potential function call overhead.

(defn sumsqr1 [s1 s2] (reduce + (map (fn [x y] (#(* % %) (- x y))) s1 s2)))

The next version uses a very straightforward implementation to take the square of the difference between x and y. I was expecting this one to be the fastest.

(defn sumsqr2 [s1 s2] (reduce + (map (fn [x y] (* (- x y) (- x y))) s1 s2)))

And finally an implementation (sumsqr3) that uses the Java pow function.

(defn sumsqr3 [s1 s2] (reduce + (map (fn [x y] (Math/pow (- x y) 2)) s1 s2)))

Strictly speaking this last function is not the same as the first two ones, since it returns a floating point number, while the other two return integers. To execute my benchmark I called the three versions repeatedly like this:

(time (dotimes [_ 1e6] (sumsqr1 (range 10) (range 1 11))))

I executed all three versions in the REPL, using Clojure 1.1, 1.2 and 1.3 Alpha 1. Results are shown in the next graph:

As you can see the sumsqr1 version is the fastest. Next is the version that uses Math.pow. The differences in execution time aren’t spectacular (about 10 %). What is more interesting to conclude is that in this particular micro benchmark, Clojure 1.2 is about 10 % faster than Clojure 1.1 and that Clojure 1.3 Alpha 1 is already 10 % faster than release 1.2. The developers are doing some great work here!