Phoenix JDBC driver and SYSTEM.CATALOG compatibility after interrupted upgrade migration

Short answer

No — an older Phoenix JDBC client should not be expected to query a catalog that a newer version has partially migrated. Phoenix performs the SYSTEM.CATALOG upgrade in place on the first connection from a newer client, and there is no forward-compatibility guarantee that older clients can read metadata written by a newer catalog version. In practice the old client fails at connection time with metadata or version incompatibility errors rather than silently degrading. There is also no documented in-place \"un-migrate\" or catalog repair utility; the supported recovery path is restoring the pre-upgrade state of all SYSTEM.* tables from HBase snapshots taken before the first new-client connection.

Likely explanation vs. confirmed facts

Well-established behavior: the catalog upgrade mutates SYSTEM.CATALOG and related tables (SYSTEM.SEQUENCE, SYSTEM.STATS, SYSTEM.FUNCTION, etc.) in place, and the client validates catalog structure at connection time. Less certain: exactly how far an interrupted migration progressed and which tables were touched — a partial migration is not a well-defined state, and detection is empirical (connection failures, exceptions from the metadata coprocessor, or catalog version messages in logs). Exact behavior, log messages, and error types vary by release, so treat the specifics below as a checklist to verify against your versions' release notes, not as universal guarantees.

Answers to your three questions

1. Older client vs. partially migrated catalog: assume incompatibility. Even a half-finished migration can leave columns or rows the old client's connection-time validation rejects.
2. Rollback without snapshots: there is no supported alternative. Do not hand-edit SYSTEM.CATALOG rows via the HBase shell — that is unsupported and risks worse corruption. If no pre-upgrade snapshot exists, the realistic options are completing the upgrade instead of rolling back, or rebuilding the cluster metadata from scratch. Snapshots taken after the upgrade started do not help.
3. Alignment: client driver, region server coprocessor JARs, and catalog version must all be the same Phoenix release. Reverting only the client JAR while region servers still run the newer coprocessor leaves the boundary inconsistent. For cross-major jumps (e.g., 4.x → 5.x), the required HBase version and coprocessor deployment model also change, so rollback may involve HBase-side reversion too.

Recovery sequence

1. Stop new connections (block clients or disable the tables) to prevent further mutation.
2. Review Phoenix and HBase logs from the first new-client connection to gauge how far the migration got.
3. Restore all SYSTEM.* tables — not just SYSTEM.CATALOG — from the pre-upgrade snapshot. Restoring a subset leaves the cluster inconsistent.
4. Redeploy the matching old coprocessor JARs to every region server; verify filenames/checksums match the old release exactly.
5. Rolling-restart region servers so the old coprocessors load.
6. Reconnect with the old driver and verify: list tables via JDBC DatabaseMetaData and run a representative query against a real user table before reopening to traffic.

One diagnostic detail matters here: whether a pre-upgrade snapshot of the SYSTEM.* tables actually exists. If it does not, rolling back is likely the wrong goal, and pushing the interrupted upgrade through to completion (with matching new coprocessors everywhere) is usually the safer path.