cgroup v2 memory.high and memory.max Reclamation Priority

Direct Answer

No, the kernel does not enforce a global priority that always swaps anonymous memory under a parent's memory.high before a child's memory.max triggers the OOM killer. Each cgroup enforces its own limits independently during charge. A child over its memory.max can OOM even if the parent has not reached its memory.high boundary.

Confirmed Behavior

In cgroup v2, memory.high is a best-effort throttling target. Crossing it causes the kernel to attempt reclaim in that cgroup and can stall allocating tasks, but usage may remain above the threshold. memory.max is a hard limit; a charge that cannot fit after reclaim triggers a memcg OOM in the offending subtree.

Anonymous pages are not exempt from memory.high reclaim. If swap is available and allowed by memory.swap.max, reclaim may swap anon pages. If swap is disabled or memory.swap.max is 0, anon pages are effectively unreclaimable, so the kernel reclaims clean file pages first and may still hit memory.max.

Likely Explanation for Nested Boundaries

The kernel does not choose parent soft reclaim over child hard enforcement. A parent's memory.high applies to aggregated usage and can create hierarchical reclaim pressure on descendants, but it does not cancel or override a child's memory.max. If the child's hard limit is lower than the parent's soft threshold, the child can OOM before the parent ever crosses memory.high.

Reclamation Order Is Not Absolute

There is no universal rule that swaps all anonymous memory before OOM. Reclaim follows LRU aging and cost: clean file pages may be dropped before swapping anon, dirty pages require writeback, and mlocked or protected pages are not reclaimed. memory.min and memory.low protections can also prevent reclaim of specific pages.

Verification on Modern Kernels

Exact reclaim order and throttling behavior are kernel-version sensitive. On modern stable kernels with the unified cgroup v2 hierarchy, verify with a nested test:

1. Create a parent and child where the child's memory.max is below the parent's memory.high.
2. Allocate a mix of anonymous and file-backed pages inside the child.
3. Monitor memory.current, memory.swap.current, memory.stat, and memory.events.
4. Check memory.events for high, max, oom, and oom_kill, and compare kernel OOM logs with the reported memcg path.
5. Repeat with swap enabled and with memory.swap.max=0 to confirm whether anon swap changes the OOM decision.

Missing Diagnostic Detail

If memory.min, memory.low, or memory.swap.max are set to non-default values in either the parent or child, provide those values. They change whether reclaim can succeed and therefore whether the outcome is throttling or OOM.