CPU throttling in cgroup v2 causes latency spikes under concurrent load

Short answer

There is no kernel-sanctioned PSI threshold that defines "acceptable" for a web service — the documentation deliberately leaves this to operators because tolerable stall time depends on your latency SLO. As a practical starting point, many operators treat sustained some avg10 values above roughly 10–20% on /proc/pressure/cpu, or any sustained full value above a few percent, as a signal worth investigating. Treat these as heuristic alarm levels, not targets, and calibrate them against your own baseline. And no: systemd-oomd is not a mitigation for CPU throttling at all — it acts only on memory pressure, so manual cgroup tuning is required for the latency spikes you're describing.

Separating confirmed facts from interpretation

What you can confirm directly:

• nr_throttled and throttled_usec in the service's cpu.stat increment only when the cpu.max quota is hit. If these counters rise during your latency spikes, throttling is a confirmed contributor — not a guess.
• PSI some means at least one task stalled; full means all non-idle tasks stalled simultaneously. For a latency-sensitive service, rising full on CPU is the stronger warning sign.

What is interpretation: the 10–20% / ~5% figures. They are community heuristics, not documented thresholds. A batch worker can tolerate far higher stall averages than a p99-sensitive API. Measure your PSI averages during a known-good period and alert on deviation from that baseline rather than on an absolute number.

Balancing CPUQuota and MemoryMax

The common failure mode is setting CPUQuota from a capacity-planning spreadsheet rather than from observed burst behavior. Under concurrent load, requests arrive in bursts; a quota set at exactly your average utilization guarantees throttling on every burst. A workable approach:

1. Measure peak CPU usage during a representative load test with the quota temporarily removed or set very high.
2. Set CPUQuota with headroom above that peak — for example, if the service peaks at 1.6 cores on a 4-core box, a quota around 200% leaves burst room while still bounding runaway behavior. Note that on cgroup v2, cpu.max enforcement happens per 100 ms period by default, so even brief bursts within a period can throttle; generous headroom matters more than the long-run average suggests.
3. Set MemoryMax modestly above the observed working set (plus headroom for allocator spikes), and consider MemoryHigh as a softer early-warning limit that throttles reclaim before a hard OOM.

Verify each change rather than trusting it:

# Before and after a load test
systemd-cgls /system.slice/myapp.service
cat /sys/fs/cgroup/system.slice/myapp.service/cpu.stat
# Watch pressure live during the test
cat /proc/pressure/cpu
# Apply and reload
systemctl set-property myapp.service CPUQuota=200% MemoryMax=2G

Success looks like throttled_usec staying flat across a repeated load test while p99 latency improves. If throttled_usec still climbs, the quota is still the binding constraint regardless of memory settings.

Where systemd-oomd fits

systemd-oomd monitors PSI memory pressure and kills cgroups that exceed configured memory/swap pressure limits. It is a last-resort availability mechanism for memory exhaustion — it does nothing about cpu.max throttling, and under bursty memory use it can terminate your service mid-spike, turning a latency problem into an availability problem. Keep it enabled as a safety net if you want, but treat CPU quota sizing and memory limits as the primary controls.

One caveat

This assumes a single-service slice on a dedicated host. If multiple latency-sensitive units share the same parent slice, contention can come from sibling cgroups rather than the unit's own quota — in that case, check cpu.pressure and cpu.stat at the parent slice level before tuning anything.