TeX capacity exceeded when building repeatable documentation pipelines

Addressing TeX Capacity Limits

There is no universal "maximum number of auxiliary files" that triggers a capacity error, as TeX limits are primarily based on internal memory pools (main memory, save stack, and token registers) rather than file counts. However, the token count—the total number of expanded macros and characters held in memory—is the primary trigger. When a document grows or a pipeline runs repeatedly without cleaning auxiliary files, the accumulation of cross-references and citations in .aux files can push the engine past its static memory allocation.

Engine-Specific Memory Behavior

The three engines handle memory differently, which explains why a build may succeed in LuaTeX but fail in pdfTeX:

• pdfTeX and XeTeX: Use static memory allocations defined at compile-time. If the document exceeds these limits, the engine throws a "capacity exceeded" error and halts.
• LuaTeX: Dynamically allocates memory for many of its internal structures, making it significantly more resilient to large documents and complex macros.

Deterministic Tuning for CI Pipelines

To achieve deterministic builds across all three back-ends without simply masking inefficient macro usage, follow these steps:

1. Identify the Exhausted Pool: Inspect the .log file immediately following the failure. Look for the specific pool mentioned (e.g., main memory, save stack, or token stack).
2. Modify texmf.cnf: Locate your texmf.cnf file using kpsewhich texmf.cnf. Increase the specific limit identified in the log. For example, to increase main memory:

   main_memory = 5000000

3. Rebuild Formats: Changes to texmf.cnf are not dynamic. You must run the format generation utility to apply them:

   fmtutil-syscfg

4. Implement Clean-State Builds: To prevent .aux file accumulation from affecting token counts in repeatable pipelines, ensure your CI script executes a cleanup command (e.g., rm *.aux *.log) before each fresh build cycle.

Verification and Diagnostics

To verify that the limits are applied, you can run a test compilation of a known "heavy" subset of your documentation. If the error persists despite increased limits, the issue is likely an infinite macro loop rather than a resource shortage.

Missing Diagnostic Detail: Please provide the specific "capacity exceeded" message from your .log file (e.g., is it save stack or main memory?). This determines whether you need to adjust memory allocation or refactor recursive macros.