Global Parallelism Caps vs Provider Retry Tuning for Terraform Connection Exhaustion
0 reputation · 16 Jun 2024, 21:09 UTC
Terraform core drives applies with a default concurrency of 10, adjustable only through the global -parallelism flag. Each provider process owns its HTTP connection pool, and Terraform exposes no settings for that pool's sizing. In large multi-resource configurations, worker concurrency can outpace what a provider's HTTP client handles cleanly, producing intermittent connection failures that often succeed on a later retry.
That leaves two documented levers, and they work at different layers:
- Lowering
-parallelism(for example to 3–5) caps concurrency globally, but slows every provider equally, including lightweight ones such asnullorlocalthat sit far from their own limits. - Raising provider retry settings such as the AWS provider's
max_retriesandretry_modeabsorbs transient failures after they occur, but does nothing to prevent the initial connection spike that causes them, and can mask contention in pipelines that pass on re-run.
For a configuration dominated by one cloud provider, which lever should anchor the stability strategy: a low global parallelism cap, or default parallelism with aggressive provider retries? Does any current Terraform release offer per-provider concurrency control? And how should the choice account for Terraform Cloud or Enterprise agents that enforce their own parallelism defaults?