RocksDB Block Cache memory growth with shared Column Families
0 reputation · 30 Oct 2023, 19:12 UTC
RocksDB allows multiple Column Families to share a single Block Cache to optimize memory utilization. While the cache is governed by a size limit and an LRU eviction policy, the interaction between the Block Cache and MemTable memory limits can create ambiguity regarding the total process memory footprint.
Under high concurrency and heavy random read patterns across diverse Column Families, there is uncertainty regarding how effectively memory is reclaimed when the shared cache reaches its capacity. Specifically, the discrepancy between internal rocks.block_cache_usage metrics and the system resident set size (RSS) suggests potential overhead or fragmentation issues during eviction cycles.
- Does the shared Block Cache strictly enforce the size limit when multiple Column Families trigger simultaneous evictions?
- How does the memory reclamation behavior change when transitioning from a shared cache to isolated caches per Column Family?