Packer 1.8+ HTTP Provisioner: Latency When Concurrent Builds Run
0 reputation · 20 Aug 2022, 03:44 UTC
0 reputation · 20 Aug 2022, 03:44 UTC
The HTTP provisioner in Packer 1.8+ serves templates via a Go http.Server listening on localhost:55555 by default. When multiple builders are executed in parallel, they share the same listening port, which can create contention for incoming connections. This contention manifests as noticeable latency spikes in the build logs, even though the templates themselves are small and local.
Current documentation does not address whether the provisioner’s server is shared across all concurrent builds or if each build spins up its own instance. Additionally, there is no mention of connection pooling, request deduplication, or per-build port isolation as a mitigation strategy.
To clarify the behavior and guide configuration, we need to determine the following:
33775 reputation · 20 Aug 2022, 10:40 UTC
Yes — the latency you are seeing is consistent with all builders contending for the same default HTTP port (55555). Packer's HTTP file-serving component binds one listener per Packer process, not one shared daemon, so the real failure mode is a bind conflict or queuing on a single socket when several builds start together. The practical fix is to give each concurrent build its own http_port (or let Packer pick a free port). There is no documented connection pooling, request deduplication, or automatic per-build isolation in the core provisioner as of the 1.8 line — verify against your exact version, since port-allocation behavior has changed across releases.
Confirmed behavior: the HTTP server is a plain Go http.Server started inside each Packer run, bound to http_bind_address/http_port (defaulting to localhost:55555 in many templates). Two processes cannot bind the same TCP port simultaneously, so a second build starting while the first holds 55555 will either fail to bind or, depending on timing, retry — which shows up as stalls in the log before provisioning begins.
Likely, but verify: the latency spikes mid-build (not just at startup) suggest requests are being serialized through one listener or that a retry loop is running. Packer's documentation does not state whether the socket is shared or per-build, and there is no documented pooling or deduplication layer. Small local templates rule out transfer time, so contention at bind/accept time is the plausible explanation.
Set a unique port per builder, or use a port range so Packer picks a free one:
source "null" "a" {
communicator = "none"
}
build {
sources = ["source.null.a"]
# Packer will choose a free port in this range
http_port_min = 8600
http_port_max = 8699
}
If your version only supports a fixed http_port, assign distinct values per build (55556, 55557, …) from your orchestration layer. Then verify:
Port-binding semantics changed around the 1.8.x line and may differ in newer releases; check the changelog for your installed version (packer version). If you can share whether the spike appears at provisioner startup or mid-build, that would distinguish a bind conflict from accept-queue saturation and would change whether port isolation alone is sufficient.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.