Closing the Loop: Packer Post-Processors That Upload to S3 and Kick Off a CloudFormation Update

Your Packer build finishes, the artifact is ready, and then… nothing. Someone has to copy the file to S3, edit a parameter, and click "update stack" in the CloudFormation console. That manual gap is where builds get lost, versions drift, and Friday deploys go wrong. Packer's post-processors can close that gap: the same template that builds the artifact can also publish it and trigger the deployment that consumes it.

The thesis is simple: treat post-processors as the last mile of your pipeline, not an afterthought. Chain an S3 upload with a CloudFormation stack update inside the template, and every build becomes traceable end-to-end.

What post-processors actually do

A post-processor runs after a builder finishes and receives the builder's artifact as input. Post-processors can be chained: each one consumes the previous one's output. Built-in options include manifest (records build metadata as JSON), checksum, compress, shell-local (runs a local command), and cloud-specific ones like amazon-import.

One honest caveat up front: Packer does not ship a dedicated "upload arbitrary file to S3" or "trigger CloudFormation" post-processor. The practical, well-supported pattern is to use shell-local to call the AWS CLI. That keeps everything in one template while relying on tooling you already trust.

A worked example: build, upload, deploy

This HCL template builds a compressed archive (stand-in for any artifact), uploads it to S3, then updates a CloudFormation stack whose parameter points at the new object. Run it on a machine or CI runner with the AWS CLI installed and credentials that allow s3:PutObject on the target bucket and cloudformation:UpdateStack on the stack.

packer {
  required_plugins {
    amazon = {
      source  = "github.com/hashicorp/amazon"
      version = "~> 1"
    }
  }
}

variable "artifact_bucket" { default = "my-artifacts-bucket" }
variable "stack_name"      { default = "app-stack" }
variable "version"         { default = "1.0.0" }

source "null" "example" {
  communicator = "none"
}

build {
  sources = ["source.null.example"]

  post-processor "shell-local" {
    inline = [
      "aws s3 cp ./app-${var.version}.tar.gz s3://${var.artifact_bucket}/releases/app-${var.version}.tar.gz"
    ]
  }

  post-processor "shell-local" {
    inline = [
      "aws cloudformation update-stack --stack-name ${var.stack_name} --use-previous-template --parameters ParameterKey=ArtifactVersion,ParameterValue=${var.version} --capabilities CAPABILITY_IAM"
    ]
  }

  post-processor "manifest" {
    output = "manifest.json"
  }
}

The null builder keeps the example focused on the post-processor chain; in a real pipeline you'd swap in an amazon-ebs or docker builder. The key detail is the version variable: it flows through the artifact name, the S3 key, and the CloudFormation parameter, so the stack always references exactly what this build produced.

IAM and region: where this silently breaks

Post-processors inherit the credentials and region of the Packer process. If your builder runs in us-east-1 but the stack lives in eu-west-1, the update-stack call fails — or worse, hits the wrong stack with the same name. Pin the region explicitly in the CLI commands (--region eu-west-1) rather than relying on ambient config.

Scope the IAM policy tightly: s3:PutObject on the releases prefix only, and cloudformation:UpdateStack plus cloudformation:DescribeStacks on the one stack. If the stack template creates IAM resources, the caller also needs iam:PassRole on the relevant roles — a common surprise failure.

Trade-offs worth knowing

• Tight coupling cuts both ways. A failed stack update fails the whole Packer build, which is good for traceability but means a CloudFormation drift issue blocks your artifact pipeline. Some teams split this into two pipeline stages instead.
• Large artifacts slow everything. Compress before upload, and remember S3 transfer time counts against your CI timeout.
• Name collisions. Re-running with the same version overwrites the S3 object. Use semantic versions or a git SHA, and consider enabling S3 object versioning for an audit trail.
• Debugging is opaque by default. Run packer build -debug template.pkr.hcl or set PACKER_LOG=1 to see exactly what each post-processor executed.

Verify before you trust it

After a build, check each link in the chain: aws s3 ls s3://my-artifacts-bucket/releases/ to confirm the object landed, and aws cloudformation describe-stacks --stack-name app-stack --query 'Stacks[0].StackStatus' to confirm the update reached UPDATE_COMPLETE. The manifest post-processor's manifest.json gives you a local record tying the build ID to the version you deployed.

Start with the upload step alone, verify it, then add the stack update. Automating the last mile is worth it — but only if you can see each step succeed on its own first.