Embedding Pulumi Automation API in CI/CD: Provisioning an AWS S3 Bucket from GitHub Actions

The problem: manual CLI steps break repeatable pipelines

Teams that adopt Pulumi often start with the interactive CLI—pulumi up on a laptop, approve the preview, and watch resources appear. That works for exploration, but it becomes a liability when the same stack must be created on every pull request, in a nightly test environment, or on demand for a developer sandbox. Manual runs introduce drift, forgotten approvals, and a reliance on a specific engineer’s workstation.

Pulumi’s Automation API solves this by letting you drive stack operations (preview, up, refresh, destroy) from ordinary application code. The same TypeScript, Python, Go, or .NET program that defines infrastructure can also be the entry point for a CI job, a platform service, or a CLI wrapper you build for internal teams.

Why Automation API fits CI/CD

• Headless execution – no interactive prompts, no TTY required.
• Language parity – you keep the infrastructure definition in the language your team already uses; the automation layer is just another import.
• Explicit safety gates – you can run preview(), inspect the diff, and only call up() when the plan matches policy.
• Backend flexibility – state can live in Pulumi Cloud, an S3 bucket, Azure Blob, or any supported remote backend, so you can meet compliance or cost constraints.

Worked example: create an S3 bucket from a GitHub Actions job

The following minimal project shows the complete loop: a Pulumi program that declares an S3 bucket, a tiny Automation API driver that runs preview then up, and a GitHub Actions workflow that executes the driver on every push to main.

1. Pulumi program (TypeScript)

// infra/index.ts
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const bucket = new aws.s3.Bucket("ci-bucket", {
  // versioning is a safe default for CI‑created buckets
  versioning: { enabled: true },
  serverSideEncryptionConfiguration: {
    rule: { applyServerSideEncryptionByDefault: { sseAlgorithm: "AES256" } }
  }
});

export const bucketName = bucket.id;

Run npm install @pulumi/pulumi @pulumi/aws in the infra folder. The program uses only stable Pulumi 3.x APIs (tested with Pulumi CLI 3.113).

2. Automation API driver

// automation/run.ts
import { LocalWorkspace, Stack } from "@pulumi/pulumi/automation";
import * as path from "path";

async function main() {
  // Point to the Pulumi project directory
  const workDir = path.join(__dirname, "../infra");
  const stackName = process.env.PULUMI_STACK || "ci";

  // Create or select the stack
  const stack = await LocalWorkspace.createOrSelectStack({
    stackName,
    workDir,
    // Use the same backend you configure locally (Pulumi Cloud, S3, etc.)
  });

  // Optional: set config values from environment
  await stack.setConfig("aws:region", { value: process.env.AWS_REGION || "us-east-1" });

  console.log(`=== Preview for stack ${stackName} ===`);
  const preview = await stack.preview({ onOutput: console.log });
  console.log(preview.summary.resourceChanges);

  // Fail fast if preview shows unexpected deletions
  if (preview.changeSummary.delete > 0) {
    throw new Error("Preview indicates deletions – aborting.");
  }

  console.log(`=== Applying stack ${stackName} ===`);
  const up = await stack.up({ onOutput: console.log });
  console.log(`Bucket name: ${up.outputs.bucketName?.value}`);
}

main().catch((err) => {
  console.error(err);
  process.exit(1);
});

Install the automation package: npm install @pulumi/pulumi@latest (the automation entry point ships with the core SDK). The driver reads PULUMI_STACK and AWS_REGION from the environment, making it reusable across dev, staging, and prod stacks.

3. GitHub Actions workflow

# .github/workflows/pulumi-ci.yml
name: Pulumi CI
on:
  push:
    branches: [main]
  workflow_dispatch: {}

permissions:
  id-token: write   # for OIDC AWS auth if you use it
  contents: read

jobs:
  pulumi:
    runs-on: ubuntu-latest
    env:
      PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
      PULUMI_STACK: ci
      AWS_REGION: us-east-1
    steps:
      - uses: actions/checkout@v4
      - name: Install Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "20"
      - name: Install Pulumi CLI
        uses: pulumi/actions@v5
        with:
          version: "3.113.0"
      - name: Install dependencies
        run: |
          cd infra
          npm ci
          cd ../automation
          npm ci
      - name: Configure AWS credentials (OIDC)
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsPulumi
          aws-region: ${{ env.AWS_REGION }}
      - name: Run Automation API driver
        run: |
          cd automation
          npx ts-node run.ts

Key points in the workflow:

• pulumi/actions installs the exact CLI version used locally, guaranteeing parity.
• AWS credentials are supplied via OIDC role assumption—no long‑lived keys in the repo.
• The PULUMI_ACCESS_TOKEN secret authenticates to the chosen state backend (Pulumi Cloud in this example). If you use an S3 backend, replace the token with AWS_* env vars and set PULUMI_BACKEND_URL=s3://my-pulumi-state.
• The driver runs preview first; the job fails if any deletions appear, providing a cheap safety gate before up.

Trade‑offs and limitations

Aspect	Benefit	Cost / Risk
State backend	Remote backends enable collaboration and locking.	Mis‑configured locking (e.g., missing DynamoDB table for S3 backend) causes race conditions when parallel jobs target the same stack.
Debugging	Structured logs from onOutput give visibility.	Headless runs hide interactive prompts; a failed up may leave a partially applied state that requires manual pulumi refresh or pulumi stack export to inspect.
Secrets handling	Pulumi config/secrets integrate with CI secret stores.	Accidental console.log of secret values leaks them into workflow logs—always set secret: true on config and avoid printing outputs that contain secrets.
Version drift	Pinning Pulumi CLI and SDK versions in CI eliminates surprise upgrades.	Automation API surface can change between major Pulumi releases; test upgrades in a staging pipeline before rolling to production.

Actionable next steps

1. Prototype locally – run npx ts-node automation/run.ts with a throwaway stack (PULUMI_STACK=dev) and verify the bucket appears in AWS.
2. Choose a backend – for team use, start with Pulumi Cloud (free tier) or an S3 + DynamoDB backend you control. Document the backend URL and locking mechanism.
3. Add policy checks – extend the driver to parse preview.changeSummary and reject plans that create resources outside an allow‑list (e.g., no IAM users).
4. Promote to a reusable job – package the driver as a Docker image or a GitHub Action so other repos can call uses: myorg/pulumi-automation@v1 with stack name and config as inputs.
5. Monitor drift – schedule a nightly pulumi preview (via the same driver) on each managed stack and alert on unexpected diffs.

By moving the “run Pulumi” step out of a developer’s shell and into a version‑controlled, reviewable pipeline, you gain repeatability, auditability, and the ability to hand off environment provisioning to a platform team without handing out cloud credentials.