Selecting STCS or LCS for Write‑Intensive Cassandra Tables

Problem: read latency spikes during heavy writes

Your Cassandra cluster serves a time‑series metrics table that receives a steady stream of updates. During peak write periods you observe occasional read latency spikes and uneven disk usage. The underlying cause is often the compaction strategy chosen for the table.

Thesis: matching the compaction strategy to the workload smooths latency and controls resource use

SizeTieredCompactionStrategy (STCS) and LeveledCompactionStrategy (LCS) have distinct trade‑offs. By aligning the strategy with the write‑read pattern and the frequency of deletes/updates, you can reduce write amplification, keep read latency predictable, and avoid sudden disk‑space surges.

How STCS and LCS work

STCS triggers a compaction when a set of similar‑sized SSTables accumulates. It merges them into a larger file, which minimizes write amplification but can cause large, infrequent compactions that temporarily spike I/O and disk usage.

LCS organizes SSTables into levels where each level is roughly ten times larger than the previous one. SSTables within a level are kept sorted, giving predictable read latency and lower read amplification. The trade‑off is higher write amplification and a larger on‑disk footprint because data may be duplicated across levels.

When STCS is the better fit

• Workloads with frequent updates or deletes (e.g., user‑profile tables where rows are overwritten).
• Scenarios where minimizing write amplification is more important than ultra‑steady read latency.
• Tables that experience bursty writes followed by quiet periods; STCS’s infrequent major compactions fit this pattern.

When LCS is the better fit

• Append‑only or immutable data (e.g., log‑style event tables) where reads dominate.
• Workloads that require consistently low read latency, such as serving dashboards.
• Tables with low delete rates, so tombstone accumulation across levels stays manageable.

Worked example: switching a metrics table from STCS to LCS

Assume a table metrics in keyspace telemetry that currently uses the default STCS. You want to move to LCS to achieve steadier read latency for a monitoring dashboard.

1. Verify the current strategy:

   nodetool describeflags telemetry metrics

   Run as the cassandra user (or via sudo -u cassandra nodetool ...). No special privileges beyond being able to execute nodetool are required.

2. Alter the table to use LCS:

   cqlsh -e "ALTER TABLE telemetry.metrics WITH compaction = {'class': 'LeveledCompactionStrategy'};"
   

   This command only updates the schema; existing SSTables remain in their current format.

3. Trigger a rewrite of existing SSTables to the new layout:

   nodetool upgradesstables -a telemetry metrics
   

   The -a flag upgrades all SSTables. This step initiates a major compaction, which can temporarily double the disk space used by the table and increase I/O load. Schedule it during a maintenance window or when traffic is low.

4. Monitor the change:

   • Run nodetool compactionstats before and after the upgrade to see pending tasks drop and the compaction strategy shift to LCS.
   • Inspect JMX metrics via jconsole or jmxterm: look at org.apache.cassandra.metrics.CompactionManager attributes PendingTasks, CompletedCompactions, and BytesCompacted to gauge write amplification.
   • Use cassandra-stress with a write‑heavy profile (e.g., insert ratio=0.9) for a short benchmark and compare latency and disk usage against a baseline run with STCS.

Trade‑off and limitation

Switching to LCS improves read latency predictability but raises write amplification and requires up to ~10 % more disk space on average. If your table later experiences a high rate of deletes or frequent updates, tombstones can proliferate across levels, degrading read paths and potentially triggering compaction storms. In such cases, reverting to STCS (or using a time‑windowed compaction strategy) may be necessary.

Actionable closing

1. Profile your workload: measure read/write ratio, update/delete frequency, and latency requirements. 2. Start with the default STCS, observe nodetool compactionstats and JMX metrics during peak load. 3. If read latency spikes correlate with major STCS compactions, trial LCS on a clone of the table using the steps above. 4. Validate with cassandra-stress and monitor disk usage; ensure the increased footprint stays within your capacity limits. 5. Document the chosen strategy and schedule a review after any significant change in access patterns (e.g., new delete‑heavy feature).

By aligning compaction strategy with the actual access pattern, you keep Cassandra’s write path efficient while providing the read performance your applications need.