Clojure transducers: single-pass transformations without intermediate collections

You have transformed a Clojure collection with (->> (range 100) (map inc) (filter even?) (reduce +)) and noticed a lazy sequence building up step by step. Or you want that same transformation logic to work with a core.async channel instead of a vector. The pattern that eliminates intermediate collections and decouples the transformation from the data source is a transducer.

Use a transducer when you want to compose transforming functions that run in a single pass, regardless of whether the source is a collection, a lazy seq, or a channel, and you want the result materialized into a specific collection without materializing intermediate steps.

How a transducer works

A transducer is a function that takes a reducing function and returns a new reducing function. In Clojure core, (map f) called with one argument returns a transducer: it does not immediately process data. When you compose transducers with comp, the resulting transducer can be applied to any reducing context.

(def xf (comp (map inc) (filter even?)))
;; xf is now a transducer

Calling (transduce xf + 0 (range 10)) feeds the input through the composed reducing functions in one pass accumulating with + and starting at 0.

Worked example: transduce a map/filter pipeline

(transduce (comp (map inc) (filter even?)) conj [] (range 10)) ;; [2 4 6 8 10]

Here (map inc) with one argument returns a transducer that increments each element. (filter even?) returns a transducer that keeps only even values. comp threads them so data flows left-to-right: increment then filter. The final conj reduces into a vector and [] is the initial accumulator. The result is produced in a single pass without an intermediate lazy sequence.

Equivalently (into [] (comp (map inc) (filter even?)) (range 10)) realizes the same result using into which also accepts a transducer as its second argument.

Transducers with core.async channels

Clojure channels can be parameterized with a transducer at construction so the same transformation logic applies whether you are reducing a collection or streaming data through a channel.

(def ch (chan 1 (comp (map inc) (filter even?))))
;; Values read from ch are incremented and filtered even, all within the channels pipeline.

Limits and common mistakes

• Stateful transducers such as take drop partition-all and dedupe carry per-evaluation state. Do not share such a transducer across concurrent reductions; each reduction should get its own instance.
• transduce and into are eager. If your source is infinite, wrap it in sequence or place take inside the transducer pipeline.
• (map f) with one argument is a transducer; (map f coll) is the classic lazy sequence function. Use the arity that matches your intent.
• comp reads left-to-right as data flows, which is the reverse of how comp reads for plain functions. A pipeline (comp (map inc) (filter even?)) increments first then filters.

Performance: transduce vs. ->> threading

For large collections transduce avoids allocating a lazy sequence per pipeline stage. Each map or filter in a ->> thread builds a lazy seq that realizes one step at a time increasing garbage-collection pressure. A transducer pipeline applies all steps in one reducing pass.

You can compare the two forms in a REPL:

(time (do (->> (range 1000000) (map inc) (filter even?) (reduce +))) ;; elapsed time in milliseconds
(time (do (transduce (comp (map inc) (filter even?)) + 0 (range 1000000))) ;; elapsed time in milliseconds

In practice the transducer form often completes with less allocation. Verify by inspecting bytecode or using a profiler if precise measurement is needed.

Quick verification checklist

1. Run (transduce (comp (map inc) (filter even?)) conj [] (range 10)) in a REPL; you should see [2 4 6 8 10].
2. Confirm (into [] (comp (map inc) (filter even?)) (range 10)) yields the same vector.
3. Check that (map inc) without a collection argument returns a transducer, while (map inc (range 10)) returns a lazy sequence.
4. If using a stateful step like take, place it inside the transducer only if you intend single-use; otherwise instantiate fresh for each reduction.