vtctl Rolling Upgrade vschema Consistency Gap
0 reputation · 28 May 2020, 10:06 UTC
After a Vitess rolling upgrade initiated via vtctl in current releases, a failed step can leave vschema metadata in etcd out-of-sync with the actual tablet state across shards. This drift may not surface immediately but can cause DDL failures, query routing errors, or inconsistent results for downstream consumers. A Vitess vschema restoration command has a state requirement that all tablets must be in a healthy, consistent condition; if any tablet is still processing writes or holds partial changes, the command aborts or applies modifications incompletely. Compounding the issue, there is no automated pre-upgrade health-check that validates tablet readiness, forcing operators to manually verify row-level consistency before and after the upgrade sequence.
- Under what conditions does vitess ResetSchema safely apply when some tablets report mixed vschema states?
- Can etcd vschema revisions be selectively reverted without full topology restart?
- What mechanisms exist to detect and resolve vschema drift after a failed rolling upgrade without data loss?