Ruby Enumerable#lazy: Memory-Saving Pipelines With Hidden Costs

You have a pipeline that reads millions of rows, maps each row to a struct, selects the ones that match, and then takes the first N. With eager Enumerable the chain builds three intermediate arrays before you ever look at a value. On a large file that means memory spikes and GC pauses, even though you only needed a handful of results.

The useful takeaway is that Enumerable#lazy defers work until consumption, keeping memory constant for long streams, but it changes when side effects run and adds per-element overhead that can make small collections slower.

How lazy enumerators defer evaluation

Calling .lazy on an Enumerable returns an Enumerator::Lazy wrapper. Operations like map, select, reject, take, drop and zip stay lazy when called on that wrapper. No block runs and no array is allocated until a terminal method pulls values.

Terminal methods are each, to_a, force and similar consumers. Once a terminal method is reached the pipeline is evaluated element by element. The wrapper holds the chain of operations, not the results.

This timing shift matters for side effects. In eager code a map block runs immediately. In a lazy chain the block runs only when next or each requests a value, which can surprise logging or mutation that developers expected to happen at definition time.

A concrete streaming pipeline

Consider processing a large numeric range without materializing it. The following script can be run with ruby on any 2.0+ interpreter, no special permissions required.

range = 1..1_000_000
lazy_pipe = range.lazy.map { |n| n * 2 }.select { |n| n % 3 == 0 }.take(5)

puts lazy_pipe.class
# Expected check: Enumerator::Lazy, not Array

lazy_pipe.each { |v| puts v }

The intermediate value lazy_pipe remains an Enumerator::Lazy. You can verify the deferral by inspecting class and by stepping with next:

enum = range.lazy.map { |n| n * 2 }.select { |n| n % 3 == 0 }
enum.peek

peek forces evaluation of the first element only. The mapping block is invoked only when a value is pulled, not when the chain is built. Risk: accidental materialization with to_a or each without a limiting operation will still allocate the full sequence.

Trade-offs and limitations in production

Lazy adds per-element dispatch overhead. For small collections an eager chain is typically faster and simpler to read. Use lazy when the source is large, unbounded, or when you need early termination with take or drop.

Debugging is harder because errors surface late at consumption time, not at definition time. A NoMethodError inside a map block will appear only when that element is requested.

Thread safety is not provided. Sharing a single lazy enumerator across threads can lead to inconsistent state because the internal cursor is mutated on next.

Not every Enumerable method is lazy. Some methods have eager variants or force evaluation implicitly. Review the chain for accidental terminals.

When to reach for lazy

Use lazy for streaming inputs, infinite ranges, and pipelines where you can bound consumption with take, first, or a finite each. Keep the enumerator local to one consumer and avoid storing it for reuse. Prefer eager collections for small, bounded data where clarity and speed matter more than memory.

Check the result by confirming the class of intermediate steps stays Enumerator::Lazy and by ensuring a terminal operation limits work to the needed amount.