Question
CircleCI Resource Class: Optimizing CPU Allocation for Docker‑in‑Docker Workflows
Ari Dey
0 reputation · 05 Aug 2021, 09:45 UTC
66.8K views1
Loading…
0 reputation · 05 Aug 2021, 09:45 UTC
Reduce build times for Docker‑in‑Docker workflows by selecting the most appropriate resource_class while staying within plan limits and cost constraints.
The resource_class key applies per job, but the actual CPU throughput differs between executor types (Linux, Windows, macOS). Moreover, higher classes (e.g., large, xlarge) consume more credits and may trigger OOM kills if the container exceeds its memory cap. It is unclear how these factors interact when multiple Docker‑in‑Docker jobs run concurrently.
large and xlarge classes for Docker‑in‑Docker jobs?resource_class selection with build duration and credit usage?33775 reputation · 05 Aug 2021, 14:37 UTC
circleciresource_class is set per job, and Docker-in-Docker containers share that job’s vCPU and memory. A larger class adds headroom for the daemon and sibling containers, but speedup is not linear. Single-container builds often see little difference; builds that run multiple containers in parallel benefit more.docker stats CPU hovering near the quota, and increasing throttled nanoseconds in cgroup cpu.stat. The build slows down instead of being OOM-killed.resource_class, start/end timestamps, and credit consumption for every job. Compare p50 and p95 build time plus credits per successful build across classes using CircleCI Insights or an internal dashboard. Run controlled A/B runs with identical workflows on large and xlarge.large and xlarge vary by CircleCI plan and executor; treat published specs as version-sensitive and verify them before changing config.yml.large and xlarge on your executor and plan.uptime, docker stats --no-stream, and cgroup throttled time (example path: /sys/fs/cgroup/cpu/cpu.stat; path differs on cgroup v2).large and optimize layer caching or container concurrency instead of upgrading.steps:
- run:
name: CPU throttling check
command: |
echo "Load average:"; uptime
echo "Docker stats:"; docker stats --no-stream
echo "cgroup throttled time (ns):"; cat /sys/fs/cgroup/cpu/cpu.stat 2>/dev/null | grep throttled || true
Which executor type (Linux, Windows, or macOS) and CircleCI plan are you using? This changes the resource-class mapping, pricing, and whether the steps above apply directly.
Use comments to ask for clarification. Post a solution as an answer.
33,775 reputation · 05 Aug 2021, 13:07 UTC
When optimizing Docker-in-Docker (DinD) workflows, it is critical to distinguish between CPU exhaustion and memory-related failures. While OOM events are explicit, CPU-throttling often manifests as a non-linear increase in build wall-clock time without the job failing.
To detect if you are hitting limits without an OOM event, you can monitor the pressure stall information (PSI) via /proc/pressure/cpu or inspect the cpu.stat file within cgroups. High throttled_usec values indicate that the container is being restricted by the kernel scheduler despite having available memory-resource-class capacity.
In DinD specifically, remember that scaling from large to xlarge provides more nominal vCPU, but if the bottleneck is the nested Docker daemon's I/O handling, you may see high credit consumption with diminishing returns on actual build speed.