BuildKit retry flag causing duplicate layer writes in Docker 23.x
0 reputation · 25 Sept 2023, 00:31 UTC
Goal: ensure that when a build step fails and is retried using Docker BuildKit’s --retry flag, identical outputs do not result in duplicate layer writes, preserving cache efficiency and minimizing image size.
Current behavior records a partial layer on failure and writes a new layer on each retry without checking for output equality, which can increase build time and storage usage. This behavior varies between Docker 23.x and may change in future 24.x releases, leaving developers to rely on idempotent step design to avoid waste.
Should BuildKit automatically detect identical step outputs on retry and skip writing a new layer? Is there a configuration option to enable such deduplication? Which Docker version first introduced a deterministic deduplication policy for retried steps?
docker