NixOS Generational Rollbacks and Stateful Service Schema Compatibility

Question

System Profile Immutability vs. Stateful Data
NixOS generations provide immutable snapshots of the system profile, allowing for rapid restoration of previous system states. However, this mechanism only affects the derivation of the system configuration and does not extend to stateful data stored in /var/lib or other persistent directories.
When a system service introduces a breaking schema change to its underlying data store during an upgrade, rolling back to a previous generation restores the older binary and configuration but leaves the migrated data in its newer format. This creates a mismatch where the older service version may be unable to parse or operate on the upgraded data schema.
Design Constraints

The declarative nature of NixOS ensures the system profile is consistent.
External data migrations are typically one‑way operations performed by the service at runtime.There is currently no native framework within the NixOS module system to define corresponding "downgrade" scripts that trigger during a generational rollback.

Does NixOS provide a documented method to synchronize stateful data rollbacks with system profile rollbacks? If not, what is the recommended architectural pattern for ensuring service stability when rolling back across a breaking schema change?

Tasadduq Burney · Answer

Answer NixOS does not provide a built‑in mechanism that automatically rolls back stateful data when you roll back a system generation. Confirmed facts Each NixOS generation is a symlink pointing to a derivation in the Nix store; rolling back changes only that symlink (nixos-rebuild switch --rollback). Stateful directories such as /var/lib, /etc (for mutable config), or any external mount are left untouched by a generation switch. Therefore, after a rollback the service binary and its configuration revert to the older version, but any data that was migrated by the newer version remains in the new schema. Recommended architectural pattern Backup before upgrade – Take a snapshot or dump of the service’s data directory before applying a generation that may contain a breaking schema change. Version‑aware service hooks – In the NixOS module for the service, add a preStart (or execStartPre) script that: Reads the current schema version from the data store. Compares it to the version expected by the binary about to be started. If the data is newer, either abort with a clear message or run a documented downgrade migration script. Provide explicit downgrade/migration scripts – Package the downgrade logic (e.g., SQL dump/restore, version‑specific migration tool) inside the same derivation and expose it via a systemd service or a oneshot hook that the preStart can invoke. Optional: Couple data snapshots to generations – If your storage backend supports snapshots (Btrfs, ZFS, LVM), create a snapshot of the data volume as part of the generation activation hook (system.activationScripts) and roll back that snapshot together with the generation when you issue nixos-rebuild switch --rollback. This requires custom glue code because NixOS does not do it automatically. Verification steps (after a rollback) # List generations to confirm the target nix-env --list-generations -p /nix/var/nix/profiles/system # Check service status systemctl show --property=ActiveState,SubState # Look for schema‑related failures journalctl -u -b | grep -i "schema\|version\|migration" Missing diagnostic detail To give an exact downgrade command or to verify that a specific service’s schema version is incompatible, I would need to know the name of the service (e.g., postgresql, mariadb, redis) and the version numbers involved in the upgrade that introduced the breaking change.

NixOS Generational Rollbacks and Stateful Service Schema Compatibility

System Profile Immutability vs. Stateful Data

Design Constraints

1 answer

Answer

Confirmed facts

Recommended architectural pattern

Verification steps (after a rollback)

Missing diagnostic detail

Your perspective belongs here.

1 question comment

Hooking a Data Snapshot into the Rollback Flow