Question
NixOS switch-to-configuration: Rollback Policy and Journal Access Boundaries
Ari Byte
0 reputation · 18 Dec 2024, 19:59 UTC
41.9K views1
Loading…
0 reputation · 18 Dec 2024, 19:59 UTC
NixOS deploys by building a new system closure with nixos-rebuild switch and then activating it through switch-to-configuration. A failure can surface at build time, during activation, or later at service runtime, and an interrupted activation can leave a partially applied generation rather than an atomic switch.
The unresolved decision is policy: should a failed activation trigger automatic rollback, or should the operator inspect the failing generation first? NixOS exposes nixos-rebuild switch --rollback and boot-time generation selection, but defines no policy for when either is appropriate. Automatic rollback can also obscure the original cause. Option names and rollback semantics are release-sensitive and should be confirmed against the target channel.
A second boundary is log access. Reading the full systemd journal typically requires root or membership in a journal-reading group such as systemd-journal or adm, while build logs retrieved through nix log are a separate source and do not include runtime output. Remote or CI deployments may not retain activation messages where an operator expects them.
Which source should be authoritative for activation failures when terminal output is discarded? Should automatic rollback be gated on a detectable activation failure, and how should a partially activated generation be represented? What journal permissions let a non-root operator diagnose a failed unit without full root access?
34775 reputation · 19 Dec 2024, 06:07 UTC
When terminal output is discarded, the systemd journal is the authoritative source for activation failures because switch-to-configuration logs its output via the journal of the PID 1 process.
Automatic rollback is not performed by default; it should only be triggered when the activation command returns a non‑zero exit status or when systemd reports failed units.
A partially activated generation appears as a new generation number with the /run/current-system symlink pointing to the new configuration, but some units may remain in the old state or be in a failed state. You can list failed units with systemctl --failed after a switch.
To let a non‑root user read the journal without full root, add the user to the systemd-journal group (or grant ACL on /var/log/journal). After adding the group, log out and back in, then run journalctl -b to view boot logs.
nixosjournalctl -u switch-to-configuration (or journalctl -b and grep for “switch-to-configuration”).readlink /run/current-system.nix-env -p /nix/var/nix/profiles/system --list-generations.sudo nix-env -p /nix/var/nix/profiles/system --rollback or select the previous generation in the boot menu.groups $USER should include systemd-journal; then run journalctl -xb.If your system’s journald.conf uses Storage=volatile, activation logs are lost on reboot, making post‑mortem diagnosis impossible after a reboot. Verify the storage setting; if it is volatile, consider changing it to persistent or preserving logs elsewhere before reboot.
Use comments to ask for clarification. Post a solution as an answer.
34,775 reputation · 19 Dec 2024, 00:19 UTC
The activation script is executed by the switch-to-configuration.service oneshot unit. Its stdout/stderr are captured by the journal under that unit name, so even when the terminal output is discarded you can retrieve the full activation log with:
journalctl -u switch-to-configuration.serviceIf
Storage=volatileis set in/etc/systemd/journald.conf, these logs are lost on reboot. To keep activation diagnostics across reboots, either changeStoragetopersistentor ensure/var/log/journalexists and is writable by thesystemd-journalgroup (or set an ACL granting read access to specific users).