Clojure Transducers: One Pipeline, Several Contexts

The pipeline you want to reuse

Suppose you have a transformation you apply in more than one place: keep even numbers, square them, sum the result. With Clojure's sequence functions you might write it as a threading pipeline:

(->> data
     (filter even?)
     (map #(* % %))
     (reduce + 0))

That works, but it ties the transformation to sequences. If the same logic needs to run over a core.async channel or a stream of values, you rewrite it. Each step also produces an intermediate lazy sequence, which the next step walks. For a large vector in a hot loop, that allocation and traversal can matter.

Transducers, added in Clojure 1.7 and stable since, separate the transformation from the context. The same pipeline can run over a collection, a channel, or a reducing operation without changing the transformation itself.

A transducer transforms a reducing function

The mental model that trips people up: a transducer is not a sequence. It is a function that takes a reducing function and returns a new reducing function. A reducing function is the kind of function you pass to reduce: it takes an accumulator and an input, and returns the next accumulator.

So (map inc) with one argument returns a transducer. When applied to a reducing function rf, it returns a function that increments each input before passing it to rf. (filter even?) returns a transducer that only forwards inputs satisfying the predicate.

Composition uses comp, and the order reads left-to-right in terms of data flow:

(def xf
  (comp (filter even?)
        (map #(* % %))))

Data hits filter first, then map. That is the opposite of ->> threading intuition, where the first form is applied first but written at the top. With comp, the leftmost transducer is the first to see each input. This is worth verifying in a REPL with a small literal vector before trusting it in a larger pipeline.

Worked example: sum of squares of even numbers

Run this in a Clojure REPL. The exact numbers are placeholders; substitute your own data.

(def data (vec (range 1 1000001)))

;; Sequence pipeline
(def seq-result
  (->> data
       (filter even?)
       (map #(* % %))
       (reduce + 0)))

;; Transducer pipeline
(def xf (comp (filter even?) (map #(* % %))))
(def xdc-result (transduce xf + 0 data))

Both forms should produce the same value. The sequence version builds intermediate lazy sequences for the filtered and mapped results; the transducer version feeds each element through the composed reducing function and accumulates directly. transduce walks the input once and does not retain intermediate collections.

You can also reuse xf with into when you want a collection out:

(into [] xf data)

And the same xf can be handed to core.async's chan with a transducer argument, or to eduction for a reducible view. That reuse is the main reason to reach for transducers rather than a private helper that only works on sequences.

Where transducers earn their keep, and where they do not

Transducers are eager in the sense that transduce and into drive the reduction immediately. They are not lazy sequences. If your code depends on laziness, such as consuming an infinite sequence with take, check the semantics carefully. take does have a transducer arity, but the pipeline still runs until the reducing function signals completion.

Not every sequence function has a transducer arity. Before converting a pipeline, check the function's documentation for a one-argument form. Functions like map, filter, take, mapcat, and dedupe do; others may not.

For small collections or one-off transformations, the threading pipeline is usually clearer. The performance difference is negligible at small sizes, and transducers add a conceptual step. They pay off when the pipeline is reused across contexts, when the data is large enough that intermediate allocations matter, or when the transformation sits in a hot loop.

How to check the result

First verify equivalence: run both versions on a small literal vector and compare with =. Then, if performance is the reason for the change, measure. (time ...) is a rough first look; a benchmarking library such as criterium gives more reliable numbers. Treat any speedup as environment-dependent: JVM version, data size, and workload all affect it. Do not assume a fixed factor.

For the authoritative list of transducer-returning functions and the exact semantics of transduce, eduction, and into with a transducer, check the official Clojure reference documentation for your release.

An actionable next step

Pick one pipeline that you already reuse in more than one context, or one that runs over a large collection in a hot path. Extract it into a named transducer with comp, verify it produces the same result as the sequence version on a small input, then measure before and after. If the measurement does not justify the change, keep the simpler threading version. That is a complete, low-risk way to decide whether transducers belong in your code.