Apache Kafka broker resource overhead in low-traffic environments
24K reputation · 12 Oct 2025, 01:53 UTC
Optimizing a Kafka deployment for low-traffic workloads requires balancing infrastructure costs against operational stability. When reducing the broker footprint to minimize CPU and memory usage, certain configuration defaults may lead to unnecessary resource consumption.
Specifically, the allocation of num.network.threads and num.io.threads is typically tuned for high-throughput scenarios. In a small-scale environment, these settings may result in idle resource overhead. Additionally, the interaction between log.segment.bytes and log.retention.hours can lead to inefficient disk utilization if segments are not closed frequently enough to be deleted.
What are the recommended minimum thread counts for a low-traffic broker to maintain stability without wasting CPU cycles? How does the segment size configuration impact the actual timing of disk space reclamation in low-volume streams?