Question
Pulumi Automation API preview memory growth in large stacks
Mira Lane
0 reputation · 12 Mar 2020, 01:12 UTC
37.3K views1
Loading…
0 reputation · 12 Mar 2020, 01:12 UTC
The Pulumi Automation API runs the engine in‑process and keeps the full resource state graph alive for the lifetime of a Workspace. When repeatedly creating a stack, previewing, and disposing the workspace, users observe a steady increase in resident memory that does not plateau, even after calling workspace.dispose(). The engine itself is expected to release the graph after disposal, but the API’s documented lifecycle does not guarantee that no references remain between invocations. Additionally, the design decision about whether preview results should be materialized entirely in memory before returning or streamed incrementally remains unresolved, potentially affecting peak memory for very large stacks.
To determine whether this behavior is a leak or an intentional design choice, the following aspects need clarification:
Workspace is disposed, or are there hidden references kept by the SDK?Answering these questions will help decide whether additional GC triggers or architectural changes are required for large‑stack Automation API workloads.
35275 reputation · 12 Mar 2020, 11:16 UTC
The observed memory growth during repeated preview operations in the Pulumi Automation API stems from how the engine manages its in-process state graph and the lack of explicit cleanup guarantees. Below is a breakdown of the likely mechanisms at play.
pulumiResourceGraph objects and state may remain referenced by static registries or background thread pools until garbage collection runs.Change objects before being returned, meaning peak memory scales with the number of resources in the stack.For a stack with thousands of resources, peak memory during preview is typically proportional to the size of the state graph plus the in-memory representation of all changes. While an update operation may have similar memory characteristics, the key difference is that preview does not execute resource modifications, so the working set is largely read-only state data.
workspace.dispose() call using System.gc() in your test harness to determine if the JVM can reclaim memory.ResourceGraph instances and trace their references.jcmd VM.native_memory or /usr/bin/ps to track resident memory before and after disposal.If memory growth persists after GC, consider running preview operations in separate processes or containers to isolate memory. Alternatively, batch large stacks into smaller logical units to reduce per-preview memory consumption.
Is there a specific reason your workload cannot tolerate the latency of process isolation? If not, that may be the most reliable path to stable memory usage.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.