Docker daemon concurrency limits affecting image pull latency

Question

The Docker daemon utilizes the max-concurrent-downloads and max-concurrent-uploads settings to manage the parallel transfer of image layers. By default, these limits are set to three concurrent operations. In high-density environments such as CI/CD runners or autoscaling nodes, multiple simultaneous container creations often trigger image pulls that exceed this default threshold.
While these settings specifically address the transfer phase, they can manifest as significant latency when layers are queued waiting for available concurrency slots. It remains unclear whether the bottleneck in these scenarios is strictly the daemon-side queue or if the performance degradation shifts to resource exhaustion like disk I/O during the layer extraction phase.
How does the Docker daemon prioritize layer requests when the max-concurrent-downloads limit is reached? Is there a documented method to determine if pull latency is caused by the transfer throttle or the subsequent decompression and filesystem extraction process?

Siddharth · Answer

Answering the Core Question
Priority handling: The Docker daemon enforces a global max-concurrent-downloads limit. When the limit is reached, additional pull requests are queued in a first‑in‑first‑out (FIFO) order. The daemon does not reorder newer pulls ahead of older ones; the earliest queued pull is the first to acquire a download slot.
When the queue is full

New pulls are marked as queued in the daemon event stream.
They remain idle until a download slot frees up, after which the download phase begins.

Diagnosing Transfer vs. Extraction Latency
Two distinct stages can cause a pull to finish slowly:

Transfer phase – downloading compressed layers from the registry.
Extraction phase – decompressing layers and writing them to the local filesystem.

To decide which stage dominates, follow these steps:

Check daemon configuration:
cat /etc/docker/daemon.json | jq ".max-concurrent-downloads"

Capture daemon events during a pull:
# Start a pull and record events from that point
start=$(date +%s)
docker pull --quiet busybox &
# In another terminal
docker events --filter type=daemon --since $start | grep pull

Look for events labeled queued or download. A long gap between queued and download indicates a transfer throttle.

Use plain progress output:
docker pull --progress=plain busybox

Note the timestamp when the Downloading section finishes versus when Extracting starts. A long pause between these markers signals extraction overhead.

Monitor system resources during the pull:
top -b -d 1 | grep docker
iostat -xz 1

High CPU or disk I/O after the download completes points to extraction delays.

These diagnostics work reliably on Docker 20.10 and newer. If you are on an older release, the event format may differ slightly.
What to Do Next?
Once you know which phase is the bottleneck:

Transfer‑heavy: Increase max-concurrent-downloads or use a faster registry network, but be mindful of registry rate limits.
Extraction‑heavy: Improve host storage performance, or reduce the number of layers by using slimmer base images.

Could you share the Docker daemon version you’re running? Certain older versions handle queue metrics differently, and that would affect the exact output of the commands above.

Docker daemon concurrency limits affecting image pull latency

1 answer

Answering the Core Question

When the queue is full

Diagnosing Transfer vs. Extraction Latency

What to Do Next?

Your perspective belongs here.

0 question comments