Diagnosing NixOS Activation Failures After nixos-rebuild switch

Recognizable Condition

After running nixos-rebuild switch the command aborts with an error or the system appears to reboot but a service fails to start, leaving you unable to log in or use expected functionality.

Cause & Diagnostic Table

Possible Cause	Typical Symptom / Error Message	Quick Check
Syntax error in configuration.nix (missing semicolon, mismatched brackets)	Evaluation stops early with error: syntax error, unexpected '}' or similar.	Run nix-instantiate --find-file /etc/nixos/configuration.nix; a non‑zero exit indicates a parse problem.
Store path mismatch – referenced package not in current Nixpkgs channel	Attribute missing error, e.g. attribute 'hello' missing while evaluating.	Execute nix-instantiate --eval -E 'with import {}; hello'; failure means the package isn’t available.
Hardware configuration incompatibility (kernel module, firmware)	Boot stalls or service fails with module xyz not found in journalctl -b -1.	Inspect boot.kernelModules and compare with lsmod output; missing modules indicate mismatch.
Service activation failure due to missing user/group or dependency	Service enters failed state; systemctl status <service> shows User "foo" not known or Dependency failed for ….	Check declared users/groups in configuration.nix against cat /etc/passwd and cat /etc/group; verify required packages are listed.

Ordered Diagnostic Steps

1. Validate the Nix expression – run

   nixos-rebuild test

   as root (or with sudo). This builds and activates the configuration in a temporary environment without touching the running system. If it fails, the error originates during evaluation or build.
2. Check for syntax problems** – if test fails with a parse error, run

   nix-instantiate --find-file /etc/nixos/configuration.nix

   . A clean exit (no output) means the file can be located; any output indicates a problem.
3. Verify package availability** – for each custom package or module attribute, test with

   nix-instantiate --eval -E 'with import  {}; <attribute>'

   . Replace <attribute> with the suspect name (e.g., vim). Failure points to a channel mismatch.
4. Inspect hardware relevance** – look at boot.kernelModules, hardware.opengl, etc. Compare with lsmod and lspci -k. Missing modules suggest the hardware config is inaccurate for the running machine.
5. Review service declarations** – examine services.<service> blocks. Ensure any required users/groups are declared under users.users and users.groups, and that all depended packages appear in environment.systemPackages or the service’s requires.
6. Check logs after a failed activation** – if switch appears to succeed but a service fails, run

   journalctl -b -1

   to see the boot log of the previous attempt. Look for lines containing the service name or failed.

Fixes Tied to Findings

• Syntax error – edit /etc/nixos/configuration.nix to add the missing semicolon or correct brackets. Re‑run nixos-rebuild test to confirm the error disappears before proceeding to switch.
• Store path mismatch – update the Nixpkgs channel:

  sudo nix-channel --update nixos

  or pin a specific revision in configuration.nix using builtins.fetchTarball. After updating, repeat the package‑availability check.
• Hardware incompatibility – remove or replace the offending kernel module or firmware line in boot.kernelModules (or hardware.opengl.driSupport, etc.). Re‑run nixos-rebuild test; if the test passes, proceed with switch.
• Service dependency/user issue** – add the missing user/group declaration:

  users.users.myapp = { isSystemUser = true; }; 

  or ensure the required package is listed in environment.systemPackages or the service’s dependsOn. Validate with nixos-rebuild test.

Escalation Criteria & Rollback

If after applying the above fixes the configuration still fails to activate:

• Escalate to the NixOS community (Discourse, Matrix, or GitHub) with the full error output, the relevant snippet of configuration.nix, and the Nixpkgs channel revision (nix-instantiate --eval -E '(import {}).lib.version').
• As a safety measure, you can rollback to the last known‑good generation without reinstalling:

  sudo nixos-rebuild switch --rollback

  activates the previous generation’s boot entry. Alternatively, select the earlier entry from the GRUB boot menu.

Practical Verification

After a successful nixos-rebuild switch, confirm the change took effect:

1. For a package change:

   which <command>

   should point to the new store path.
2. For a service:

   systemctl status <service>

   shows active (running).
3. For hardware:

   lsmod | grep <module>

   confirms the module is loaded.

Limitations: nixos-rebuild test and dry-activate do not exercise hardware‑specific code paths (e.g., early boot drivers). A full reboot or nixos-rebuild switch is required to validate those aspects.