Espresso Schema Management and Storage Engine: Version Garbage Collection Timing

Answer

The safe window for removing an old schema version in Espresso is defined by observed client migration progress, not by a fixed time‑to‑live. When the schema‑management service sees that no active client version still references the obsolete schema (i.e., the client‑version watermark has advanced past that schema), it signals the storage engine to retire the version.

Confirmed facts

• Espresso uses schema‑on‑read versioning so newer optional/defaulted fields can coexist with older application versions during multi‑phase rollouts.
• Retaining many concurrent schema versions adds read‑path latency because each read must resolve the applicable schema before decoding.

Likely explanation

1. The schema‑management layer maintains a reference count or lease for each schema version, incremented when a client version that could need that schema is detected (via deployment metadata or a client‑version watermark).
2. As rollouts proceed, the watermark advances; when the count for an old version drops to zero, the management service emits a retirement signal.
3. The storage engine receives this signal, decrements its internal refcount, and purges the schema version’s metadata once the count reaches zero.
4. If a late‑migrating client attempts a read after the version has been retired, the storage layer will either return defaults/nulls for fields that are still resolvable via the reader schema, or it will return a schema‑mismatch error if the reader schema can no longer be resolved. The exact behavior depends on whether the reader schema remains available in the version registry.

Steps to verify or act on this case

1. Check the schema‑management service for a metric that exposes the current client‑version watermark or active schema version registry.
2. Correlate retirement events in the logs with the point at which the watermark advances past the target schema version.
3. In a staging cluster, deliberately run a read with an old client after the schema version has been marked retired and observe whether the response contains defaults/nulls or a schema‑mismatch error.

If you cannot locate a client‑version watermark or equivalent metric, please confirm whether your Espresso deployment exposes such a signal; its presence changes the recommendation from "wait for watermark advance" to "inspect alternative retirement triggers".