Automatic tablet splitting versus manual pre-splitting for range-sharded tables with skewed write distribution
0 reputation · 11 Jul 2021, 04:22 UTC
0 reputation · 11 Jul 2021, 04:22 UTC
For a range-sharded YSQL table keyed on a monotonically increasing identifier, early write traffic concentrates on a small number of tablet leaders until tablet splitting engages. The design goal is to achieve even write distribution across nodes from table creation without incurring unnecessary per-tablet resource overhead.
Two documented approaches address this constraint: automatic tablet splitting, which reactively divides growing tablets under version-sensitive gflag defaults, and manual pre-splitting via SPLIT INTO N or SPLIT AT VALUES, which distributes leaders immediately at the cost of up-front schema design and potential per-tablet memory and compaction overhead.
The concrete constraint involves a skewed key distribution where a minority of tenants generate the majority of writes, raising questions about whether reactive splitting can keep pace or if explicit split points on hot key ranges are preferable.
Additionally, YSQL transaction-isolation-level mapping onto YugabyteDB's snapshot and serializable model has evolved across releases, meaning application retry logic assumptions may silently break after upgrades.
Should automatic tablet splitting under default gflags engage quickly enough to outweigh the per-tablet overhead of manual pre-splitting for a given workload? For skewed range-sharded keys, are explicit SPLIT AT VALUES on hot tenant IDs more effective than SPLIT INTO N? And does re-splitting an already-populated table require a lightweight online operation or a table rebuild?
A thoughtful contribution can make all the difference. Be the first to share one.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.