Strict vs Lazy Lists: Which Reduces Peak Memory for Large Collections?

Question

In a data‑processing pipeline that builds large collections, the choice between strict list construction (using bang patterns) and the default lazy evaluation directly impacts memory consumption and CPU overhead. The goal is to decide whether to enforce strictness on each element to reduce heap churn or to retain laziness to keep immediate CPU work low.
The constraints are that the list may contain tens of millions of elements, the application runs on a long‑lived server, and the compiler flags are set to -O2. Profiling data for the current implementation shows high peak memory but acceptable CPU time. However, it is unclear whether the memory spikes are due to retained thunks or to the cost of building the list.
The unresolved decision is whether strictness annotations will lower peak memory without introducing significant CPU overhead. We need evidence from profiling and a clear trade‑off analysis.
1. How much does a bang pattern on the list constructor reduce peak memory for a 10 million element list at -O2?2. What is the measurable increase in CPU time when strictness is applied?3. Does the GHC optimizer fuse the strict constructor with downstream folds, mitigating the CPU cost?

Zoha · Answer

Direct answer
A bang pattern or strict constructor field does not reliably reduce peak memory for a ten-million-element list compiled with GHC -O2. If the pipeline retains the full list, strictness mainly moves allocation earlier and can increase peak residency by forcing elements that laziness would never evaluate. If the pipeline consumes and discards elements, a strict left fold over a lazily generated list usually keeps residency constant without changing the list constructor. Exact megabyte savings and CPU impact cannot be predicted from the source alone.

Likely explanation versus confirmed facts
Likely explanation: The high peak memory probably comes from retained cons cells, unevaluated element thunks, or elements kept alive after the fold, rather than from the cost of building lazy cells. Strictness eliminates per-element thunks but shifts their cost forward; it does not automatically free the list spine.

Confirmed facts for GHC at -O2:

A strict constructor such as data List a = Nil | Cons !a !(List a) forces the head to weak head normal form when the cell is evaluated.
Standard foldr/build fusion targets the built-in lazy list type []; a custom strict list type generally does not participate in those rewrite rules.
-O2 enables inlining and simplifier passes but does not guarantee constant space, fusion, or lower peak residency.
CPU impact is workload-dependent: strictness can remove later thunk indirection or force unnecessary evaluation, so it may be neutral, faster, or slower by a workload-specific margin.

Steps for this case

Build both variants with ghc -O2 -prof -fprof-auto -rtsopts.
Run each with +RTS -s -hc -hy -hd at ten million elements and compare maximum residency, total allocation, and GC time.
Inspect the retainer profile (-hd) to see whether memory is held in thunks, cons cells, or retained elements after consumption.
Check Core with -ddump-simpl to confirm whether downstream folds fuse or re-traverse the list.

Missing diagnostic
Provide a heap or retainer profile showing what is alive at peak residency. That detail changes the recommendation: if the list is retained after the pipeline, strictness is unlikely to help; if elements are dropped during streaming, foldl' over a lazy list is usually the simpler fix.

Strict vs Lazy Lists: Which Reduces Peak Memory for Large Collections?

1 answer

Direct answer

Likely explanation versus confirmed facts

Steps for this case

Missing diagnostic

Your perspective belongs here.

0 question comments