Scatter-Gather vs. Lookup Shards for Global Secondary Indexing

In Vitess, managing queries that lack the primary sharding key typically requires a choice between scatter-gather routing via VTGate or the implementation of a dedicated lookup shard to maintain a global secondary index.

Scatter-gather routing provides maximum query flexibility by broadcasting requests to all tablets, but it introduces significant tail latency and CPU overhead as the number of shards increases. Conversely, using a lookup shard enables targeted routing for secondary keys, reducing the load on the cluster at the cost of additional write amplification and the need to maintain consistency between the primary and lookup shards.

When designing for a growing cluster where predictable latency is a priority but write throughput is high, the trade-off between these two routing strategies becomes critical.

• Which approach better preserves tail latency stability as the shard count scales beyond ten tablets?
• At what threshold of write volume does the overhead of a lookup shard outweigh the latency benefits of avoiding scatter-gather?