Using LLVM ThinLTO for Incremental C++ Builds

Problem

When a C++ codebase needs whole‑program optimization but developers also require fast incremental links, traditional LTO (‑flto) forces the linker to re‑optimize every translation unit on each rebuild, making incremental builds prohibitively slow.

Takeaway: LLVM ThinLTO (‑flto=thin) keeps most work local to each compile step, emits per‑module bitcode and summaries, and performs a lightweight thin‑link step that distributes imports before the final native link, giving most of the LTO benefit while preserving near‑native incremental times.

Requirements

The goal is whole‑program optimization with acceptable incremental link time. This means:

• Optimization across translation units (inlining, devirtualization, etc.)
• Incremental builds should reuse most of the work already done for unchanged sources.
• The toolchain must stay within a single LLVM major version to avoid summary incompatibility.

Smallest Suitable Design

Keep the front‑end unchanged. Add the ThinLTO flag to clang/clang++ for both compile and link steps:

# Compile each translation unit (run in the build directory, user needs read/write access)
clang++ -flto=thin -O2 -c src/foo.cpp -o build/foo.o
clang++ -flto=thin -O2 -c src/bar.cpp -o build/bar.o

The build system must then invoke a thin link step that processes the summaries and produces a combined bitcode module:

# Thin link (run after all objects are built, same permissions as compile)
clang++ -flto=thin build/foo.o build/bar.o -o build/prog.thin.bc

Finally, the backend codegen step compiles the combined bitcode to native objects and links them:

# Backend codegen and final link (parallelizable)
clang++ -flto=thin build/prog.thin.bc -o build/prog

This design adds only two extra invocations (thin link and backend codegen) while leaving the compile command unchanged.

Trust/Data Boundaries

Trusted artifacts are the source files and the build graph that drives clang. Untrusted inputs are third‑party object files or libraries that lack bitcode summaries. ThinLTO treats bitcode and summaries as build intermediates; they must be produced by the same LLVM major version as the thin link and backend steps. Mixing versions can cause the thin link to silently fall back to a no‑summary mode, losing optimization without an error.

Operational Checks

To verify that ThinLTO is active:

• Scan the build log for the thin link invocation (look for "-flto=thin" and a temporary .bc file).
• After the build, run llvm-nm build/prog | grep -E 'internal|external' and confirm that symbols expected from devirtualization or inlining appear (e.g., vtable entries resolved).
• Compare binary size: a ThinLTO build should be smaller than a plain -O2 build and close to a full -flto build.
• Measure incremental link time: touch one source file and re‑run the link step; the wall time should be comparable to a non‑LTO incremental link (typically a few seconds for medium projects).

Risks: If the thin link step fails due to missing summaries, the linker will emit a warning and proceed without optimization. Treat any warning about "summary missing" as a failure to apply ThinLTO.

Failure Modes

• Version skew: Compiling with LLVM 16 and thin linking with LLVM 17 can cause summary mismatch, leading to silent fallback or link errors.
• Missing summaries: If any object file is built without -flto=thin (e.g., a third‑party library), the thin link cannot import its summaries and will skip optimization for that translation unit.
• Memory pressure: The backend codegen step runs parallel workers; large modules can exceed available RAM, causing OOM kills or swapping.

When the Design Changes

Re‑evaluate the approach if:

• Cross‑language whole‑program optimization is required (e.g., Rust + C++), which needs full LTO to merge LLVM IR across language front‑ends.
• Deterministic builds are mandated and ThinLTO’s nondeterministic summary ordering cannot be tolerated.
• Maximum optimization for tiny release artifacts is paramount and link time is no longer a concern, making full -flto preferable.

In those cases, replace the thin link/backend codegen pair with a single -flto link step, accepting longer incremental links in exchange for stronger cross‑module optimizations.