Packer Amazon EBS Builder and AWS API: shutdown_stop vs shutdown_terminate behavior
0 reputation · 07 Mar 2024, 12:09 UTC
0 reputation · 07 Mar 2024, 12:09 UTC
When utilizing the Amazon EBS builder in HashiCorp Packer (HCL2), the lifecycle of the temporary instance is managed via specific shutdown configurations. The goal is to ensure a predictable state for the instance upon the completion of the build process to avoid orphaned resources or unexpected billing.
There is an ambiguity regarding how Packer interacts with the AWS API when transitioning from the provisioning phase to the image creation phase. Specifically, the behavior of the shutdown_stop and shutdown_terminate options may vary depending on the instance state and the specific cloud environment configuration.
shutdown_stop: Intended to stop the instance before creating the AMI.shutdown_terminate: Intended to terminate the instance after the AMI is successfully created.Does the shutdown_stop setting guarantee a clean filesystem sync before the AMI snapshot is triggered? In scenarios where shutdown_terminate is used, does Packer verify the AMI availability before issuing the termination call to the AWS API?
34775 reputation · 08 Mar 2024, 00:02 UTC
Packer amazon-ebs does not snapshot after shutdown. The AMI is created first, then the builder instance is shut down. shutdown_behavior controls only the final EC2 API call that Packer makes after the image is ready.
StopInstances after the guest OS shutdown sequence. The instance moves to stopped, the instance ID and attached EBS volumes persist, and you can start it later for inspection.TerminateInstances after the image is ready. The instance moves to terminated and is unrecoverable. EBS volumes are removed according to each volume's delete_on_termination setting.Packer waits for the AMI to reach available before it proceeds to the shutdown step. Termination is not issued until image creation is confirmed complete.
shutdown_behavior with values terminate default and stop. This setting determines the final EC2 API call performed after AMI creation.terminate, Packer calls TerminateInstances. The instance is terminated and EBS volumes are removed per their delete_on_termination flag.stop, Packer calls StopInstances after OS shutdown. The instance remains stopped with volumes attached and can be started later.The clean filesystem sync depends on the guest OS shutdown, not on the EC2 stop/terminate call itself.
shutdown_command runs inside the guest OS before Packer invokes the EC2 StopInstances or TerminateInstances API. A graceful sync therefore depends on the OS completing shutdown and the communicator timeout settings.shutdown_stop does not guarantee a clean filesystem sync before the AMI snapshot is triggered because the snapshot is triggered by CreateImage while the instance is still running, before any shutdown.stop is commonly used for debugging or reusing the builder host to inspect build artifacts without relaunch. terminate is the default for clean up and cost avoidance.shutdown_behavior. Example:
source "amazon-ebs" "example" {
shutdown_behavior = "stop" # or "terminate"
shutdown_command = "..."
}
StopInstances or TerminateInstances events for that instance ID.delete_on_termination flag to confirm persistence expectations for stop vs terminate.To give a precise recommendation for your environment, what Packer version and shutdown_behavior value are you using, and do you have a shutdown_command configured? That changes whether a graceful OS sync is attempted before the final API call.
Use comments to ask for clarification. Post a solution as an answer.
34,775 reputation · 07 Mar 2024, 16:31 UTC
To build on the previous point regarding shutdown_behavior, it is important to note that switching from the default terminate to stop introduces a specific dependency on the IAM policy assigned to the Packer execution identity.
While terminate relies on ec2:TerminateInstances, using stop requires the ec2:StopInstances permission. If this permission is missing, the build may successfully create the AMI but fail during the final cleanup phase, potentially leaving the temporary instance running and incurring unnecessary compute costs.
ec2:StopInstances is granted if using shutdown_behavior = "stop".