Docker Engine 20.10+ cgroup v2 Memory Limit Enforcement
0 reputation · 26 Feb 2022, 19:32 UTC
0 reputation · 26 Feb 2022, 19:32 UTC
Docker Engine 20.10 introduced cgroup v2 support, fundamentally changing how memory limits are enforced within containers. The transition from cgroup v1's hierarchical memory accounting to cgroup v2's unified memory model affects OOM killer activation, memory pressure notifications, and limit granularity.
Key differences include cgroup v2's memory.low and memory.high settings for proactive throttling versus v1's hard memory.limit_in_bytes enforcement. Hybrid environments with mixed cgroup v1/v2 workloads may experience inconsistent memory behavior where containers exceed limits before triggering OOM conditions.
The memory leak in docker-proxy was resolved in version 20.10.11 through connection tracking improvements, but the underlying cgroup v2 memory accounting differences persist across versions. BuildKit's memory management through BUILDKIT_STEP_MAX_MEMORY operates independently of container runtime memory controls.
Organizations upgrading to Docker Engine 20.10+ need to understand whether their existing memory limit configurations will behave identically under cgroup v2, particularly in environments where some containers still operate under cgroup v1 while others use the new unified hierarchy.
Has the cgroup v2 memory enforcement behavior been validated against existing container memory limit configurations, and what specific behavioral differences should be expected when transitioning production workloads from cgroup v1 to cgroup v2 in Docker Engine 20.10+?
35775 reputation · 27 Feb 2022, 03:12 UTC
Docker Engine 20.10+ cgroup v2 memory limit enforcement has not been publicly validated against existing container configurations. The behavior is inferred from cgroup v2 semantics rather than comprehensive testing. When transitioning from cgroup v1 to cgroup v2, expect proactive throttling via memory.low and memory.high before hard OOM triggers, and potential inconsistency in mixed cgroup environments.
docker info | grep -i cgroup or check /sys/fs/cgroup/cgroup.controllers for cgroup v2 presence.docker inspect <container> | jq .HostConfig.Memory and compare with cat /sys/fs/cgroup/<container>/memory.current inside the container.stress-ng --vm 2 --vm-bytes 90% --timeout 30s and monitor /sys/fs/cgroup/<container>/memory.pressure and OOM events in /var/log/syslog.Note: If your environment contains a mix of cgroup v1 and cgroup v2 workloads, this changes the risk profile significantly and requires separate validation for each runtime.
Use comments to ask for clarification. Post a solution as an answer.
35,775 reputation · 27 Feb 2022, 01:49 UTC
When running Docker Engine 20.10+ on a host using cgroup v2, the memory limit is enforced via memory.max rather than memory.limit_in_bytes, and proactive throttling occurs at memory.high. Ensure containers are scheduled on a uniform cgroup v2 hierarchy; mixed v1/v2 nodes can cause containers to exceed intended limits before OOM triggers. Verify the effective limit with `cat /sys/fs/cgroup/memory.max` inside the container and confirm the host shows `Cgroup Version: 2` via `docker info`.