Cassandra TWCS with TTL: Designing Time-Series Partitions That Expire Cleanly

The problem: TTL alone does not reclaim space quickly

Time-series tables in Cassandra often use TTL to expire old readings. With the default LeveledCompactionStrategy (LCS) or SizeTieredCompactionStrategy (STCS), expired rows are not removed immediately. Compactors must rewrite surviving SSTables, merging live data with tombstones. On large tables this creates sustained write amplification and delays disk reclamation until a compaction happens to include the expired data.

TimeWindowCompactionStrategy (TWCS) changes the model. Instead of compacting by size or level, it groups SSTables into fixed time windows and only compacts SSTables that fall inside the same window. Once an entire window is older than the TTL, Cassandra can drop those SSTables without rewriting them. The result is predictable expiration and far less background I/O.

Match the partition to the window

TWCS works best when the partition key contains a time bucket that aligns with the compaction window. A common pattern is to bucket by day:

CREATE TABLE sensor_readings (
    sensor_id uuid,
    day_bucket bigint,
    event_time timestamp,
    event_id timeuuid,
    value double,
    PRIMARY KEY ((sensor_id, day_bucket), event_time DESC, event_id)
) WITH CLUSTERING ORDER BY (event_time DESC, event_id ASC)
    AND compaction = {
        'class': 'TimeWindowCompactionStrategy',
        'compaction_window_unit': 'DAY',
        'compaction_window_size': '1'
    }
    AND default_time_to_live = 2592000; -- 30 days

Here day_bucket is the Unix epoch day, calculated as event_time / 86400000. Keeping one day per partition bounds partition growth and aligns the data with TWCS's default one-day window. The clustering order event_time DESC makes recent-first reads and time-range scans efficient.

Target roughly 100 MB per partition. If a single sensor produces more than that in a day, add an hour_bucket to the partition key and keep the window size at one day.

TTL, gc_grace_seconds, and window alignment

Set default_time_to_live at the table level so every row expires uniformly. Avoid per-row TTL overrides that are shorter than the table TTL; they do not change when the SSTable becomes eligible for drop and can create confusion during debugging.

gc_grace_seconds controls how long Cassandra waits after a tombstone is created before it can be physically removed. For TWCS, set it to at least the maximum hint window (commonly three hours) plus your maximum repair interval. Setting it too low risks resurrecting deleted data if a node is down longer than the grace period; setting it too high delays space reclamation.

In Cassandra 5.0, compaction_window_start_offset lets you shift window boundaries. If your business day starts at 06:00 UTC, align the offset so windows do not split a logical day across two SSTable groups. Check the 5.0 release notes for the exact syntax before deploying.

What breaks: late writes and mixed strategies

TWCS does not compact across windows. A late write—caused by clock skew or a backfill job—lands in an old window and creates a small, isolated SSTable. That file will not merge with anything until it expires, wasting disk and adding read amplification. Use monotonic timestamps where possible, or accept a small buffer of late data.

Do not mix TWCS with LCS or STCS on the same table. TWCS disables minor compaction across windows, so a table that also receives frequent UPSERTs or counter updates will accumulate tombstones without the rewrite behavior needed to purge them. For mutable rows, stay with LCS or STCS.

Worked example and verification

After creating the table above, insert a reading and confirm the strategy is active. Run the following on a cqlsh prompt connected to any node:

INSERT INTO sensor_readings (sensor_id, day_bucket, event_time, event_id, value)
VALUES (uuid(), 20293, toTimestamp(now()), now(), 23.4);

To inspect compaction behavior, run nodetool compactionhistory on the database node. This command requires JMX permissions typically granted to operators or DBAs. Look for entries that show whole-window drops rather than partial merges. Use nodetool tablestats keyspace_name.table_name to check that PendingTasks for TWCS stays near zero; spikes in EstimatedRemainingTasks should occur only when a new window opens.

For a definitive check on an expired window, run sstablemetadata on the SSTable files for that window. In Cassandra 4.x and 5.x, look for Estimated droppable tombstones at or near 100 percent and an SSTable level of 0. If an old window still contains live data or a non-zero droppable ratio, a late write or misaligned TTL is the likely cause.

If you need to reclaim space immediately—for example, before decommissioning a node—run TRUNCATE keyspace_name.table_name; in cqlsh. This requires ALTER or DROP permissions on the keyspace and is destructive. Dropping a table with a long TTL can leave orphaned SSTables on disk until the TTL expires, so truncation is the safer operational path. Follow with nodetool cleanup on the node to remove unnecessary replicas.

Trade-offs and a practical checklist

• Write amplification vs. expiration latency: TWCS avoids rewriting old windows, but larger windows reduce file count at the cost of keeping expired data longer.
• Partition size: Buckets that exceed roughly 100 MB degrade read latency and repair streaming. Sub-partition by hour when necessary.
• Multi-DC consistency: Keep TTL and compaction settings identical across data centers. Run incremental repair regularly; --full repair is unnecessary for TWCS tables because anti-entropy only matters for overlapping windows.

Before rolling out, verify with a short-lived test table: set TTL to one hour, insert data across three days, and confirm that only the current window compacts while older windows drop after TTL plus gc_grace_seconds. If that holds, the production table will expire predictably.