Question
TTL expiration behavior with merge operators in RocksDB compaction
Neel Sage
0 reputation · 08 Apr 2025, 19:04 UTC
52.4K views0
Goal
Determine whether RocksDB removes TTL‑expired keys before applying a merge operator during compaction, and what impact this interaction has on write amplification and memtable usage.
Constraints / Uncertainty
- The documentation states that TTL expiration is performed lazily during compaction or reads, but does not specify the order relative to merge‑operator resolution.
- Merge operators may rewrite a key’s value based on the existing and new operands; if an expired key is still present when the merge runs, the resulting value could be unexpected.
- Current releases (6.0+) fixed a memtable leak related to TTL, but no explicit guarantee exists for merge‑operator interactions.
Given these gaps, the following questions remain unresolved:
- When a column family has both a TTL setting and a merge operator, does RocksDB guarantee that an expired key is filtered out before the merge operator is invoked during compaction?
- If the expired key is not filtered out first, what value does the merge operator produce, and does this cause extra write amplification or unnecessary memtable retention?
- Are there any configuration options (e.g., compaction style, rate limiter) that can mitigate undefined behavior when TTL and merge operators are combined?