NATS JetStream store compatibility during version downgrades

Answer First

When a NATS server binary that is older than the one that created the JetStream store starts up, it will not load the store. The server performs a version check on the JetStream domain directory, compares the on‑disk store version with the version it knows how to read, and aborts initialization with a fatal error if the on‑disk store is newer.

Confirmed Facts

• The JetStream domain directory contains a store_version file that records the format version used by the last server that wrote to the store.
• On startup, nats-server reads this file and verifies that the stored version is not newer than the version it supports.
• If the check fails, the server logs a message such as FATAL: JetStream store version 2.1 is newer than server version 2.0 – aborting and exits before any streams are opened.
• Downgrading after a store‑format‑changing release is not supported; the store may become permanently unreadable if an older binary is forced to start.
• Because Raft metadata resides in the same domain store, a mismatch also prevents a node from joining the cluster correctly.

Likely Explanation (Version‑Specific)

If a new binary writes data using a changed layout, an older binary will usually fail early during the store‑open phase rather than corrupting data. The exact failure mode can vary by release, but the mismatch detection is designed to be safe.

Practical Steps for Your Scenario

1. Locate the JetStream domain directory (default store/jetstream inside the server data directory).

2. Open the store_version file (or inspect the store/jetstream/.metadata directory) to read the on‑disk version number.

3. Start the older nats-server with the --store flag pointing to that directory and capture the log output.

4. Verify that the startup log contains a fatal store‑version mismatch message before any Stream initialization lines appear.

5. If you must recover the data, upgrade to the newest supported binary first, then perform a clean downgrade only if the release notes explicitly state that the downgrade path is safe.

Missing Diagnostic Detail

To give you the most accurate guidance, could you share the exact NATS server version numbers involved (e.g., 2.10.0 upgrading to 2.11.1) and whether the store‑format change was documented in the release notes?