Cap Aggregate Memtable Memory in RocksDB with a Shared WriteBufferManager

One column family can still consume all RAM

You run one RocksDB instance with multiple column families—for example, a write-heavy event log and a read-heavy user cache. Each family gets its own write_buffer_size, so it seems memory is bounded. It is not. Per-family limits apply to each family individually; they do not add up to a global ceiling. One family can fill RAM with active and immutable memtables while the others compete for what remains.

A memtable is the in-memory buffer for recent writes. An immutable memtable is a full memtable waiting to be flushed to disk. A column family is a named keyspace inside one RocksDB instance with its own memtables, SST files, and compaction. An SST file is the sorted, immutable table on disk that results from a flush.

What a shared WriteBufferManager bounds

A WriteBufferManager tracks the combined size of active and immutable memtables across every column family that references it. When the combined total reaches the byte ceiling you set, RocksDB applies backpressure: new writes stall until a flush frees memory. Each family still keeps isolation—separate keyspace, separate SST files, separate compaction—but they share one memory budget.

This fits mixed workloads in a single instance, such as per-user namespaces, multi-tenant embeddings, or separating write-heavy logs from read-heavy caches. Per-family write_buffer_size still acts as a soft target for each family, but the manager enforces the hard global ceiling.

Worked example: two families under one manager

Run this inside your application process that links RocksDB. The process needs read, write, and create permission on the database directory at <db_path>.

rocksdb::WriteBufferManager wbm(256ULL * 1024 * 1024); // shared 256 MB ceiling

rocksdb::ColumnFamilyOptions cf_opts;
cf_opts.write_buffer_manager = &wbm;
cf_opts.write_buffer_size = 64ULL * 1024 * 1024; // per-family target

std::vector<rocksdb::ColumnFamilyDescriptor> column_families;
column_families.emplace_back(rocksdb::kDefaultColumnFamilyName, cf_opts);
column_families.emplace_back("event_log", cf_opts);
column_families.emplace_back("user_cache", cf_opts);

std::vector<rocksdb::ColumnFamilyHandle*> handles;
rocksdb::DB* db = nullptr;
rocksdb::Status status = rocksdb::DB::Open(options, "", column_families, &handles, &db);

Replace <db_path> with your actual directory. The write_buffer_manager pointer must remain valid for the lifetime of the database; if the manager object is destroyed first, behavior is undefined.

Expected check: while writing, query the DB-level property rocksdb.cur-size-all-mem-tables. It should approach but not persistently exceed the shared ceiling. Risk: when the ceiling is reached, writes on any family sharing the manager may block or return a busy status until a flush completes, which can spike tail latency across families.

Trade-offs and limitations

• Stall propagation. A slow flush in one column family can stall writes in another because they share the same memory budget. Monitor write stall conditions if latency matters.
• Metadata overhead. Each column family adds MANIFEST and metadata overhead. Avoid creating thousands of families without load testing.
• Version variance. Property names, default limits, and stall semantics vary by RocksDB release. Verify the WriteBufferManager API against your linked version's headers and release notes before production use.
• Durability is separate. WAL settings such as WAL size and TTL are configured independently and are not affected by the buffer manager.

How to verify the ceiling is enforced

1. Open the database with two column families under one WriteBufferManager set to a modest cap.
2. Write to one family until rocksdb.cur-size-all-mem-tables approaches the cap.
3. Confirm that writes block or return a stall status, and that process resident memory stops climbing instead of growing unbounded.
4. Allow a flush to complete, then confirm memtable size drops and writes resume.

If you need to roll back this change, close the database handles and redeploy with the previous per-family-only configuration. Deleting the database directory is destructive and should only be done when the data is no longer needed.

Bottom line

Use per-family buffer sizes for local tuning, but add a shared WriteBufferManager when you need a hard ceiling on aggregate memtable memory. Verify against your RocksDB version, watch for cross-family stalls, and keep WAL durability settings separate.