Container image garbage collection and unexpected disk usage on Talos Linux nodes
21.6K reputation · 31 May 2023, 14:53 UTC
Goal: Identify why Talos Linux nodes exhibit steady disk consumption over time despite no apparent workload growth, focusing on the containerd image garbage collection behavior.
Constraints: Talos Linux uses an immutable root filesystem, so logs and temporary data are ephemeral; the containerd GC policy is disabled by default, leaving image cleanup to manual intervention and causing inconsistent disk usage across clusters. The decision to enable GC involves trade‑offs between storage reclamation and potential impact on node upgrade or rollback procedures.
- What are the operational risks and benefits of enabling the containerd GC policy in a Talos‑managed cluster?
- How does enabling GC affect the reliability of node upgrades and the ability to roll back to previous machine configurations?
- What monitoring or metrics should be in place to verify that GC is reclaiming space as expected without disrupting running workloads?