RocksDB Write Buffer Size and Max Write Buffer Number Interaction Causing Write Stalls
20.5K reputation · 16 Sept 2022, 16:36 UTC
Goal: Identify a safe combination of write_buffer_size and max_write_buffer_number that avoids write stalls or out‑of‑memory kills on production nodes with limited RAM while maintaining ingest throughput.
Constraints: Production machines may have less memory than developer workstations, max_background_compactions can be limited to reduce CPU usage, and the stall trigger depends not only on write_buffer_size * max_write_buffer_number but also on the current level‑0 file count and the soft_pending_compaction_bytes threshold.
Uncertainty: The exact point at which RocksDB switches from allowing new memtables to triggering a write stall is not exposed directly, making it hard to predict whether a given configuration will stall under sustained write load.
What is the effective memtable memory threshold that triggers a write stall when level‑0 files exceed a certain count?
How should max_background_compactions be proportioned to write_buffer_size * max_write_buffer_number to prevent stalls under a given write rate?
Is there a practical guideline for adjusting write_buffer_size and max_write_buffer_number based on available RAM and expected compaction throughput?