Pulumi ESC Architecture: Environment Config and Secret Distribution Across Stacks

The problem

Teams need environment‑specific values and secrets separated from Pulumi program code, with deterministic previews and auditable access across stages.

Requirements

• Separate code from environment data; programs must be identical across dev, staging and prod and only vary by configuration.
• Keep secrets out of source control and out of Pulumi state in plaintext; store and reference secrets in ESC.
• Provide auditability of who read which config, when a secret rotated, and which stack version consumed it.
• Enforce least‑privilege per team and stage; a dev identity must not read prod secrets.

Smallest suitable design

One ESC environment per deployment stage. The environment holds non‑secret values and secret references, not raw secrets in the program.

name: prod
values:
  region: us-east-1
  appName: payments
secrets:
  dbPassword:
    type: string
    # value set via ESC UI or API, not committed

Pulumi programs consume configuration exclusively via Pulumi.Config and do not hardcode values.

// TypeScript example
import * as pulumi from "@pulumi/pulumi";

const cfg = new pulumi.Config();
const region = cfg.require("region");
const appName = cfg.require("appName");
const dbPassword = cfg.requireSecret("dbPassword");

Link the stack to the ESC environment at deploy time. Run from a CI workspace or developer shell with access to the Pulumi backend and ESC.

# Preview using ESC environment
pulumi preview --stack prod --config-url pulumi://.../esc/prod

Required permissions: identity token with read access to the named ESC environment, and write access to the target Pulumi stack and state backend. Risk: running preview with an overly permissive identity can expose secret metadata in logs or outputs.

Trust and data boundaries

• ESC is the configuration authority; it owns schema, values, secret references, and access policies.
• Pulumi service or self‑hosted controller is the secret custodian; it fetches values from ESC during plan and apply and keeps them in memory.
• Stacks are consumers; they receive resolved values via Pulumi.Config and must not log or output secret values.
• Identity provider is the root trust anchor; ESC policies are evaluated against IdP identities and Pulumi stack identities.

Boundary rule: secrets must never be written to state in plaintext and must remain within the ESC‑to‑runtime boundary. Do not pass secrets to stack outputs, do not print Config values, and avoid interpolating secrets into resource names or tags.

Operational checks

• Validate ESC environment schema before deploy; a CI step can load the environment and verify required keys and types.
• Monitor stack preview diffs for config drift; changes in ESC values appear as config changes in preview.
• Audit ESC access logs and secret rotation events; review who accessed which environment and when secrets rotated.
• Enforce least‑privilege ESC roles per team using ESC policies that restrict read access to intended stacks and identities.

Practical verification steps:

• Review Pulumi program to confirm it reads configuration exclusively via Pulumi.Config and ESC references and does not hardcode values.
• Inspect ESC environment definition for separation of non‑secret values and secret references, and verify access policies restrict read to intended stacks and identities.
• Perform a preview deployment in a non‑production stack after rotating a secret in ESC to confirm propagation without state mutation.
• Check operational dashboards for ESC access audit logs and Pulumi stack update history to validate drift detection and least‑privilege enforcement.

Failure modes

• ESC unavailability blocks previews and updates; treat ESC as critical path and monitor its availability.
• Stale cached config causing drift; clear config cache and re‑run preview.
• Accidental secret promotion across environments; prevent by using separate environments and policy deny for cross‑environment writes.
• State lock contention in multi‑writer scenarios; coordinate writers via CI pipeline serialization.
• Secret metadata leakage; Pulumi state contains references to config keys; avoid logging or outputting Config values.

When to change the design

• Regulatory requirements demand HSM‑backed keys or specific secret residency; migrate to an external vault with Pulumi ESC vault integration.
• Secret volume exceeds ESC policy limits or operational overhead; consider a dedicated secrets manager.
• Need for cross‑cloud secret replication or advanced key management; introduce an external vault as the secret source and reference it from ESC.

Limitations: ESC secret values are only as secure as the identity provider and access policies protecting the ESC environment; misconfigured policies can expose secrets to unintended stacks. Behavior can differ between Pulumi SDK versions and self‑hosted vs SaaS ESC.