Decoupling Infrastructure from Configuration with Packer Sources

Problem: Rigid Image Templates

Many teams write Packer templates where the cloud provider settings (like instance size, region, and AMI IDs) are hardcoded alongside the installation scripts. This creates a maintenance headache: if you want to build the same software image for both AWS and Azure, or even just for two different AWS regions, you end up duplicating the entire build logic. When a package version changes, you must update every single template file.

Thesis: Use Source Blocks to Separate 'Where' from 'What'

The HCL2 syntax in Packer introduces a critical architectural distinction between the source block and the build block. By defining the infrastructure requirements in a named source and the configuration logic in a build block, you can reuse the same provisioning steps across multiple platforms or environments without duplicating code.

How Source and Build Blocks Interact

In Packer, a source block defines the machine's origin—the base image, the cloud provider, and the hardware specifications. It answers the question: "What is the raw virtual machine I am starting with?"

The build block then references one or more sources. Within this block, you define provisioners (like shell scripts or Ansible) that install software. It answers the question: "What do I need to install on this machine to make it useful?"

This decoupling allows for a "one-to-many" relationship: one set of provisioning scripts can be applied to multiple sources (e.g., an Ubuntu source for AWS and an Ubuntu source for Google Cloud Platform).

Worked Example: Multi-Region Image Pipeline

The following example demonstrates how to define two different regional sources but use a single build process to install a monitoring agent across both.

variable "region_east" { type = string; default = "us-east-1" }
variable "region_west" { type = string; default = "us-west-2" }

source "amazon-ebs" "east_base" {
  region        = var.region_east
  instance_type = "t3.micro"
  ami_name      = "monitoring-agent-{{timestamp}}-east"
  source_ami_filter { 
    filters = { name = "ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*.gz" }
    most_recent = true
    owners      = ["099720109477"]
  }
  ssh_username = "ubuntu"
}

source "amazon-ebs" "west_base" {
  region        = var.region_west
  instance_type = "t3.micro"
  ami_name      = "monitoring-agent-{{timestamp}}-west"
  source_ami_filter { 
    filters = { name = "ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*.gz" }
    most_recent = true
    owners      = ["099720109477"]
  }
  ssh_username = "ubuntu"
}

build {
  sources = [
    "source.amazon-ebs.east_base",
    "source.amazon-ebs.west_base"
  ]

  provisioner "shell" {
    inline = [
      "sudo apt-get update",
      "sudo apt-get install -y prometheus-node-exporter"
    ]
  }
}

Execution and Verification

Run these commands from your terminal in the directory containing the .pkr.hcl file:

1. Initialize: Run packer init . to install the amazon-ebs plugin.
2. Validate: Run packer validate . to ensure the HCL2 syntax is correct.
3. Build: Run packer build .. This requires AWS credentials with permissions for ec2:RunInstances, ec2:CreateImage, and ec2:TerminateInstances.

Verification: After the build, check the AWS Console in both us-east-1 and us-west-2. You should see two distinct AMIs, both containing the prometheus-node-exporter package.

Trade-offs and Limitations

While decoupling is powerful, it introduces a dependency on image parity. If you use a source for Ubuntu and another for CentOS, a shell provisioner using apt-get will fail on the CentOS build. To solve this, you must either use a platform-agnostic configuration tool like Ansible or use conditional logic within your scripts to detect the OS.

Additionally, running multiple sources in a single build block triggers parallel builds by default. If your CI/CD runner has limited resources or your cloud account has strict API rate limits, this can lead to RequestLimitExceeded errors.

Actionable Closing: Refactor Your Templates

To move toward a more maintainable image pipeline:

• Identify repeated provisioner blocks across your different templates.
• Extract the infrastructure settings (region, instance type, base AMI) into separate source blocks.
• Consolidate the installation logic into a single build block that references those sources.
• Use packer validate to ensure your references are correct before triggering a full cloud build.