Choosing the Right Cassandra Compaction Strategy: STCS, LCS, or TWCS?

Problem: Compaction Choices Impact Cluster Health

In a production Cassandra cluster, the compaction strategy you pick can make or break your read latency, write throughput, and disk usage. A strategy tuned for a write‑heavy workload can turn into a read bottleneck when your application starts fetching data, and vice‑versa. The root of the issue is that each compaction strategy trades write amplification, read amplification, CPU, and I/O for one another.

Thesis: Map Workload Characteristics to the Correct Strategy

By aligning your workload’s write rate, read pattern, and data retention needs with the right compaction strategy—Size‑Tiered (STCS), Leveled (LCS), or Time‑Window (TWCS)—you can keep latency predictable, storage efficient, and cluster resources balanced.

1. What Each Strategy Does

All compaction strategies merge SSTables (immutable on‑disk files) but differ in how they select which SSTables to merge and how they size the resulting file.

• Size‑Tiered (STCS): Groups SSTables by size. When a threshold is reached, all files in the tier are merged into a larger file. This keeps write amplification low but can produce many overlapping SSTables, raising read amplification.
• Leveled (LCS): Maintains a hierarchy of levels where each level holds SSTables of roughly equal size. Merges happen within or across levels to keep files non‑overlapping. This yields consistent read performance at the cost of higher write amplification and CPU usage.
• Time‑Window (TWCS): Compacts data in fixed time windows (e.g., 1 hour). Recent data stays in small windows for fast reads; older windows are compacted aggressively. Ideal for time‑series workloads with heavy reads on recent data.

2. When to Use Each Strategy

Workload Type	Read Pattern	Write Pattern	Recommended Strategy
Heavy Writes, Light Reads	Random, infrequent	High write throughput, low latency	STCS
Read‑Intensive, Moderate Writes	Frequent, range queries	Balanced writes	LCS
Time‑Series	Recent data hot, older data cold	Write bursts, TTLs common	TWCS

3. Practical Decision Steps

1. Profile your workload: Use nodetool cfstats and JMX metrics to see write latency, read latency, and SSTable count.
2. Check table size: LCS is automatically enabled for tables > 100 GB in Cassandra 4.0+. For smaller tables, you can force LCS via ALTER TABLE ... WITH compaction = { 'class': 'LeveledCompactionStrategy' }.
3. Consider TTL and compaction thresholds: TWCS works best when TTLs are used; set compaction_window_size and compaction_window_unit appropriately.

4. Worked Example: Switching a Keyspace from STCS to LCS

Assume you have a keyspace analytics with a table events that has grown to 200 GB and now exhibits high read latency. You decide to switch to LCS.

# Connect to the node via CQL shell
cqlsh

# Verify current compaction strategy
SELECT * FROM system_schema.tables WHERE keyspace_name='analytics' AND table_name='events';
# Look for the "compaction" column.

# Alter the table to use LeveledCompactionStrategy
ALTER TABLE analytics.events WITH compaction = {
    'class': 'LeveledCompactionStrategy',
    'sstable_size_in_mb': '64'
};

# Verify the change
SELECT * FROM system_schema.tables WHERE keyspace_name='analytics' AND table_name='events';

After the change, Cassandra will begin scheduling LCS compactions. Monitor the impact:

• Read latency: nodetool cfstats analytics.events should show a drop in ReadLatency over the next few hours.
• Write latency: Expect a slight increase; keep an eye on WriteLatency and CompactionThroughputMBPerSec.
• SSTable count: nodetool cfstats should show a reduced number of overlapping SSTables.

5. Trade‑offs & Limitations

• LCS: High CPU and I/O during compaction. If your nodes are under‑provisioned, you may see cluster instability. Mitigate by throttling compaction throughput in cassandra.yaml (compaction_throughput_mb_per_sec).
• STCS: Overlapping SSTables can balloon disk usage. Monitor sstable counts and run nodetool cleanup when needed.
• TWCS: Requires careful window sizing. Too small windows create many SSTables; too large windows reduce read benefit.
• Automatic LCS for large tables can be overridden, but manual tuning may be necessary for optimal performance.

6. Actionable Checklist

1. Run nodetool cfstats to capture baseline read/write latency and SSTable count.
2. Profile write patterns and TTL usage.
3. Select strategy using the table above.
4. Apply the strategy via ALTER TABLE or cassandra.yaml if global.
5. Monitor nodetool compactionstats, cfstats, and JMX metrics for at least 24 hours.
6. Adjust compaction thresholds or window sizes if latency or storage deviates from expectations.

Conclusion

Choosing the right compaction strategy is a balancing act. Size‑Tiered is great for write‑heavy workloads with loose read requirements, Leveled delivers steady reads at the cost of more CPU and write amplification, and Time‑Window shines for time‑series data with hot recent windows. By profiling your workload, applying the appropriate strategy, and monitoring key metrics, you can keep Cassandra’s latency predictable and its storage footprint in check.