Proxmox ZFS Pool DEGRADED After Reboot – Diagnostic Guide

Recognizable condition

A local ZFS storage pool on a Proxmox VE node reports DEGRADED, FAULTED or UNAVAIL after a system reboot. VMs that rely on the pool stop, show I/O errors, or cannot start. The Proxmox web UI marks the storage with a red warning, and the shell shows the pool state as DEGRADED or UNAVAIL.

Diagnostic overview and common causes

Sign	Possible cause
VMs stop, I/O errors	Pool state DEGRADED/FAULTED/UNAVAIL
Web UI shows red warning	Device path changed due to HBA re‑enumeration
dmesg shows missing device messages	Physical disk offline or failed
zpool history missing import entry	Pool exported manually and not auto‑imported
Pool ONLINE but VMs stall	Insufficient RAM causing ZFS ARC pressure

Ordered checks

1. zpool status -v – run on the node shell (root). Verify the pool name, its state, and each vdev’s condition. Expected: pool listed as DEGRADED, FAULTED or UNAVAIL. Risk: none if run as root.
2. dmesg | grep -i 'ata\|sas\|zfs' – look for link errors, device reset messages, or ZFS import warnings after boot. Expected: no recent “device not found” lines for the missing disk.
3. ls -l /dev/disk/by-id – compare the symlinks for the disks that belong to the pool with the entries shown in zpool status -v. Expected: stable by‑id paths match the vdev devices.
4. cat /etc/pve/storage.cfg – confirm the pool name and the device path used in the configuration. Expected: the path references a persistent by‑id identifier, not a transient /dev/sdX name.
5. zpool history – review recent import events. Expected: an entry showing automatic import after reboot; absence may indicate missing cache file.
6. free -m – check available memory. Expected: sufficient RAM to avoid ARC pressure; low memory can mimic IO stalls while the pool stays ONLINE.

Fixes tied to findings

• Device name changed – Export the pool using the stable by‑id path, then import it again:

zpool export 
zpool import -d /dev/disk/by-id/ 

Update /etc/pve/storage.cfg to use the same by‑id entry. Risk: temporary service interruption during export/import; ensure no VMs are actively writing.
• Physical disk offline or failed – Replace the faulty drive, then run:

zpool replace  /dev/disk/by-id/

Monitor resilver progress with zpool status. Risk: resilver is I/O intensive; schedule during a maintenance window and watch VM performance.
• Pool not auto‑imported – Ensure the import service is enabled and run a manual import:

systemctl enable zfs-import.service
zpool import -a

Verify the pool appears ONLINE in both shell and web UI. Risk: if the cache file is corrupted, re‑import may fail; check /var/lib/zfs/import for remnants.
• ARC pressure suspected – Adjust the ARC limit and verify memory headroom:

echo 'zfs_arc_max/0 4G' >> /etc/systemd/zfs.conf.d/99-arc.conf
systemctl restart zfs-import.service

Check zpool iostat for reduced stall rates. Risk: lowering ARC may affect cache efficiency; monitor after change.

Escalation criteria

If after the above steps the pool remains FAULTED, multiple vdevs are degraded, resilver stalls for more than a few hours, or zpool status shows data integrity errors, escalate:

1. Collect zpool history, zpool status -v output, and relevant kernel logs.
2. Verify hardware connectivity (cables, HBA status) and run vendor diagnostics on the suspect disks.
3. Consider offline import with -f only after confirming no hardware fault, as it can mask underlying issues.

Escalate to Proxmox support or the hardware vendor with the gathered logs before performing further manual repairs.

Verification after remediation

1. Run zpool status -v on the node shell and confirm the pool state is ONLINE.
2. Check that /dev/disk/by-id symlinks used by the pool match the entries in zpool status.
3. Start a test VM from the pool and execute a short backup job. Successful completion indicates IO paths and snapshot creation work.

These steps provide a clear, repeatable process to diagnose and resolve a DEGRADED or UNAVAIL ZFS pool on Proxmox VE without risking data loss.