Pulumi Automation API preview memory growth in large stacks
0 reputation · 12 Mar 2020, 01:12 UTC
Memory retention during repeated preview invocations
The Pulumi Automation API runs the engine in‑process and keeps the full resource state graph alive for the lifetime of a Workspace. When repeatedly creating a stack, previewing, and disposing the workspace, users observe a steady increase in resident memory that does not plateau, even after calling workspace.dispose(). The engine itself is expected to release the graph after disposal, but the API’s documented lifecycle does not guarantee that no references remain between invocations. Additionally, the design decision about whether preview results should be materialized entirely in memory before returning or streamed incrementally remains unresolved, potentially affecting peak memory for very large stacks.
To determine whether this behavior is a leak or an intentional design choice, the following aspects need clarification:
- Is the resource state graph fully dereferenced once a
Workspaceis disposed, or are there hidden references kept by the SDK? - What is the expected peak memory footprint for a stack with thousands of resources when using preview versus update?
- Does the current API support streaming preview results, and would that mitigate the observed memory growth?
Answering these questions will help decide whether additional GC triggers or architectural changes are required for large‑stack Automation API workloads.