Looping: for vs. doseq [Update Aug 2012: This article describes a version of Clojure before 1.0, but all the details related to macros and lazy seqs are still true in Clojure 1.5a3.] Newcomers to Clojure are often surprised to find that for is not a flow-control mechanism, but instead a lazy-generator. This may trip you up if you want side-effects: (defn count-and-print-words [text] (let [words (re-seq #"\S+" text)] (for [word words] ; not what I want (println word)) (count words))) user=> (count-and-print-words "What's that blue thing?") 4 No words are printed, because nothing uses the return value of for and so, being lazy, it does nothing at all. What you want is doseq : (defn count-and-print-words [text] (let [words (re-seq #"\S+" text)] (doseq [word words] (println word)) (count words))) user=> (count-and-print-words "What's that blue thing?") What's that blue thing? 4 But for has a lot of power that's been missing from doseq , specifically nesting loops, :while , and :when . In case you're not familiar with these, here's a quick example that just (inefficiently) lists the factors of the Fibonacci numbers under 100: (def fibs (lazy-cat [0 1] (map + fibs (rest fibs)))) (def some-factors (for [fib fibs :while (< fib 100) fac (range 2 fib) :when (zero? (rem fib fac))] [fib fac])) user=> some-factors ([8 2] [8 4] [21 3] [21 7] [34 2] [34 17] [55 5] [55 11]) Isn't that the most beautiful definition of fib? Alas, it's not my invention. Anyway, if I want to be able to have nesting, :while , and :when in an imperative, non-lazy loop, I'll have to write a macro. A good way to start is with an example of calling your macro. Start with the simplest useful example: (my-doseq [i (range 10)] (prn i))

The first rule of writing macros I heard somewhere that the first rule of writing macros is: don't write a macro. This is a cute way of reminding you to only write a macro when it's necessary. In this case it's necessary because I want my-doseq to be able to define new locals ( i in the above example). If my-doseq were a function and i wasn't defined yet, the example wouldn't even compile. So here are some ideas that I've found useful when writing macros: Don't write a macro (perhaps a function would do or the macro already exists).

Write an example usage (start with a simple one).

Expand your example usage by hand.

Use macroexpand when your macro doesn't work.

when your macro doesn't work. Experiment at the REPL

Break up complicated macros into smaller functions.

Our own looping macro I'll demonstrate all these as we go along. Write out by hand the code the macro should produce for this example: (loop [i-seq (seq (range 10))] (when i-seq (let [i (first i-seq)] (prn i) (recur (rest i-seq))))) Try that out to see that it prints 0 through 9, just like it should. Note that without the call to seq , passing in an empty collection (instead of range ) would cause (when i-seq ...) to succeed when it shouldn't. It's an important detail to remember -- use seq to get a false value for an empty collection. If you don't understand the above loop, take a moment to walk through it -- it's the foundation all the following examples are built on. A macro for this simple case is pretty straightforward: (defmacro my-doseq [[sym init] & body] `(loop [sq# (seq ~init)] (when sq# (let [~sym (first sq#)] ~@body (recur (rest sq#)))))) The hand-coded loop is in this macro almost verbatim. The backquote is what makes this work, and enables the placeholder magic: sq# for a temporary local name, ~ to insert a parameter, ~@ to splice in a parameter that's a seq. This isn't terribly impressive, though -- it's almost a straight copy of how doseq is already defined. Next up: nested loops.

Nested loops First, the example usage: (my-doseq [i (range 3) j (range 3)] (prn i j)) Expanded by hand: (loop [i-seq (seq (range 3))] (when i-seq (let [i (first i-seq)] (loop [j-seq (seq (range 3))] (when j-seq (let [j (first j-seq)] (prn i j) (recur (rest j-seq))))) (recur (rest i-seq))))) I'm actually not sure if this one easier to expand by hand or to just write the macro to do it for you. The pattern to notice, though, is that where our earlier expansion had the body, this one has another loop. What we want is a function that produces the loop part, but which can be called again if there's a nested loop: (defmacro my-doseq [seq-exprs & body] (let [emit (fn emit [[sym init & rest-exprs]] `(loop [sq# (seq ~init)] ; same loop as before (when sq# (let [~sym (first sq#)] ~(if rest-exprs ; more rest-exprs? (emit rest-exprs) `(do ~@body)) (recur (rest sq#))))))] (emit seq-exprs))) The main loop is recognizable from the previous version, but now it's in its own local function so it can be called again. Where it used to simply splice in ~@body , it now has an if to either recurse (if there's another nested loop) or in the terminal case splices in ~@body like it did before. The use of `(do ~@body) instead of just body may be a bit surprising, so let's take a moment to look at why. It shows up in a ~ expression inside a let body, so let's strip it down to just that and see what goes wrong with a plain body : user=> (defmacro tmp [& body] `(let [] ~(when true body))) nil user=> (macroexpand '(tmp (prn 1) (prn 2))) (let* [] ((prn 1) (prn 2))) Can you spot what's wrong with that expansion? What if you run it: user=> (tmp (prn 1) (prn 2)) 1 2 java.lang.NullPointerException (NO_SOURCE_FILE:0) Both the prn 's get run, but then the first one's return value is at the head of a list, so Clojure tries to invoke that return value. The prn function always returns nil, and the attempt to invoke nil causes the NPE. There a couple ways to solve this, one of which is to splice the whole body into a do : user=> (defmacro tmp [& body] `(let [] ~(when true `(do ~@body)))) nil user=> (macroexpand '(tmp (prn 1) (prn 2))) (let* [] (do (prn 1) (prn 2))) By putting the word do at the head of the list, we get the behavior we want.

Implementing the :when filter So now that nested loops are working, next up is support for :when and :while . The main difficulty here is that the number of items our emit function has to consume each iteration depends on how many of these options follow the init value: (my-doseq [prod (range 13) fac (range 2 prod) :when (zero? (rem prod fac))] (prn prod fac)) One approach to this kind of problem is to assume, for the moment, that the input to my-doseq was open to change. What format would make it easiest to write the emit function? Perhaps: [{:name prod :init (range 13)} {:name fac :init (range 2 prod) :when (zero? (rem prod fac))}] By grouping the name, initial expression, and all options together for each nesting level of the loop, the macro could be written: (defmacro my-doseq [seq-exprs & body] (let [emit (fn emit [[bind-map & rest-binds]] `(loop [sq# (seq ~(:init bind-map))] ; changed to use :init (when sq# (let [~(:name bind-map) (first sq#)] ; changed to use :name ~(if rest-binds (emit rest-binds) `(do ~@body)) (recur (rest sq#))))))] (emit seq-exprs))) This works, but is very awkward to call... but that's a problem we can solve later. For now let's make :when work, by inserting an if clause that defaults to true if no :when is given: (defmacro my-doseq [seq-exprs & body] (let [emit (fn emit [[bind-map & rest-binds]] `(loop [sq# (seq ~(:init bind-map))] (when sq# (let [~(:name bind-map) (first sq#)] (if ~(or (:when bind-map) true) ; new "if" clause ~(if rest-binds (emit rest-binds) `(do ~@body))) (recur (rest sq#))))))] (emit seq-exprs))) If no :when clause is given, such as with the prod loop shown earlier, that part of the macro will expand to: (if true (loop...)) This is slightly inefficient, but completely removing the if would make the macro significantly more complex. Instead we'll leave it in and rely on the JVM to optimize it away at runtime.

Implementing the :while limiter A minimal example using :while : (my-doseq [{:name fib :init fibs :while (< fib 40)}] (prn fib)) That's really a terribly verbose syntax. I promise we'll fix it soon. To get the :while part working, a when clause is inserted to skip both the body and the recur when the test expression is false: (defmacro my-doseq [seq-exprs & body] (let [emit (fn emit [[bind-map & rest-binds]] `(loop [sq# (seq ~(:init bind-map))] (when sq# (let [~(:name bind-map) (first sq#)] (when ~(or (:while bind-map) true) ; new "when" clause (if ~(or (:when bind-map) true) ~(if rest-binds (emit rest-binds) `(do ~@body))) (recur (rest sq#)))))))] (emit seq-exprs))) Try that out to see that it works.