Automating AWS Golden Images with Packer HCL2

The Problem: Configuration Drift in Base Images

Manual image creation leads to "snowflake" servers—instances that are uniquely configured and impossible to reproduce. When security patches or middleware updates are required, updating every running instance individually is inefficient and error-prone. The solution is a Golden Image: a pre-configured, hardened Amazon Machine Image (AMI) that serves as the single source of truth for all auto-scaling groups and new deployments.

Prerequisites

• Packer Installed: Version 1.7.0 or later to ensure full HCL2 support.
• AWS Credentials: Configured via ~/.aws/credentials or environment variables (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY). Do not hardcode these in the configuration.
• Network Access: A VPC with a public subnet or a VPN/Direct Connect that allows SSH (port 22) access from the machine running Packer to the temporary build instance.

Defining the Golden Image Pipeline

Packer uses HashiCorp Configuration Language (HCL2) to define the build process. The workflow follows a linear lifecycle: launching a source instance, provisioning software, shutting down the instance, and registering the AMI.

# Example packer-ami.pkr.hcl
variable "ami_name" {
  type    = string
  default = "web-server-golden-image"
}

source "amazon-ebs" "ubuntu" {
  ami_name      = "${var.ami_name}-${formatdate("YYYY-MM-DD", timestamp())}"
  instance_type = "t3.micro"
  region        = "us-east-1"
  source_ami    = "ami-0c55b159cbfafe1f0"
  ssh_username  = "ubuntu"
}

build {
  sources = ["source.amazon-ebs.ubuntu"]

  provisioner "shell" {
    inline = [
      "sudo apt-get update",
      "sudo apt-get install -y nginx",
      "sudo systemctl enable nginx",
      "sudo apt-get clean"
    ]
  }
}

Critical Engineering Decisions

1. Managing Image Bloat

Every package manager leaves behind cache files (e.g., /var/cache/apt). If these are not cleared before the snapshot, the resulting AMI is larger than necessary, increasing storage costs and slowing down instance boot times. Always include a cleanup command as the final step of your shell provisioner.

2. Security Group Configuration

The amazon-ebs builder creates a temporary instance. If you do not specify a security group that allows SSH access from your current IP, the build will hang and eventually time out. Use the ssh_interface or security_group_id attributes to ensure connectivity.

3. Handling Initial Boot with boot_command

For images that do not support cloud-init or require manual user creation before SSH is available, use the boot_command. This sends keystrokes to the instance console via AWS, allowing you to set passwords or configure networking before Packer attempts to connect via SSH.

Execution and Verification

Run these commands from the terminal in the directory containing your .pkr.hcl file.

1. Validate Syntax: Run packer validate . to check for HCL2 structural errors. This prevents the build from failing halfway through due to a typo.
2. Execute Build: Run packer build .. This triggers the launch of the EC2 instance and runs the provisioners.
3. Verify AMI Registration: Use the AWS CLI to confirm the image exists:

   aws ec2 describe-images --filters "Name=name,Values=web-server-golden-image*"

4. Functional Test: Launch a single instance from the new AMI and verify that the software (e.g., Nginx) is running and accessible.

Rollback and Cleanup

Because packer build creates an AMI and a snapshot, a failed build may leave orphaned resources. If a build fails, Packer generally attempts to terminate the temporary instance, but you should manually check for any remaining temporary security groups or snapshots in the AWS Console to avoid unnecessary costs.