Another design sketch for Clojure coverage tool

By: on June 20, 2010

Previously (here) I sketched out a design that went some way to being a coverage tool for Clojure code. The result was a macro that when used to define a function, resulted in an instrumented function that tallied every time the code was called. This time I am going to sketch the beginnings of a design that doesn’t use a macro.

To start with here is a slight re-write of the structure I used last time to record the coverage:

(def coverage-records (ref nil))

(defn add-record [fn-name record]
  (dosync (alter coverage-records assoc (str fn-name) record)))

(defn get-record [fn-name]
  (get @coverage-records (str fn-name)))

(defn covering [form fn-name]
  (dosync (alter (get-record fn-name) assoc form 0)))

(defn- inc-map [map key]
  (if (contains? map key)
    (assoc (dissoc map key) key (inc (get map key)))

(defn inc-coverage [form fn-name]
  (dosync (alter (get-record fn-name) inc-map form)))

This code uses a single map to store a coverage record, another map, for each function that is being measured. Functions are provided for adding records and incrementing the coverage.

I am using the same set of functions to wrap and instrument the s-expressions that define the function, with a few minor modifications. I am indexing the s-expressions with a number so as to distinguish between any identical s-expressions within the function. This set of functions is still incomplete since there are lots of Clojure code that will get wrapped incorrectly, I will be rectifying this sometime in the near future.

(declare wrap-seq)
(defn wrap
  [form idx fn-name]
    (seq? form) (
      let [key (str idx ":" form)]
        (covering key fn-name)
        (list 'do `(inc-coverage ~key ~(str fn-name)) (wrap-seq form idx fn-name)))
    :else form))

(defn- indexed [coll] (map vector (iterate inc 0) coll))

(defn wrap-seq [coll count fn-name]
  (for [[idx elt] (indexed coll)] (wrap elt (+ count idx) fn-name)))

The previous article used a macro to generate Clojure code that was instrumented, this time I will load the source code for the function, wrap it and then evaluate it. This is done by the get-source function in clojure.contrib.repl-utils combined with a call to format and load-string, My code that interprets the source code is fragile and needs more work, it won’t handle functions with multiple bodies for example, but it demonstrates the principle. I am using a structure to hold the instrumented function and the coverage record for this function.

(defstruct wrapper :wrapped-fn :coverage-record)

(defn wrap-fn
  [f cv-rec]
  (let [s (read-string (get-source f))
        fn-name (first (drop 1 s))
        args (first (drop 2 s))
        body (last s)]
    (add-record fn-name cv-rec)
    (load-string (format "(fn %s %s)" args (wrap body 0 fn-name)))))

(defn wrap-function [f]
  (let [coverage-record (ref nil)]
    (struct wrapper (wrap-fn f coverage-record) coverage-record)))

So now we can try the code out at the REPL:

user=> (def x (wrap-function 'test1))   
user=> x
{:wrapped-fn #, :coverage-record #}

Here I have wrapped the function test1 and we can see the coverage structure returned, consisting of the wrapped function and the coverage record, keyed by the single s-expression with a count of 0. If I now use the wrapped function like this and examine the coverage structure the count should increment to 1.

user=> ((:wrapped-fn x) 1 2)            
user=> x
{:wrapped-fn #, :coverage-record #}

So we now have a partially functional coverage tool and no macros have needed to be written. To complete this tool I just need to tidy up the wrapping and source code reading functions and provide some sort of binding macro so that we can call the fn with

(test1 1 2)

instead of

((:wrapped-fn x) 1 2)

. Hopefully, in my next blog entry I will have completed it!



  1. Mike D says:

    Nice idea! I started working on some code coverage tools a couple months ago also. My approach was fairly similar, but my code also expands macros as it walks the source code. By expanding macros, you can cut down on the number of special syntax forms your instrumenting code needs to handle. I haven’t had much time to work on it lately, but here it is if you want to take a look at it:

    The more interesting stuff is in instrument.clj

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>