Proxmox VE High Availability: Configure Automatic VM Failover

Quick Answer: Automatic VM Recovery After Node Failure

If a Proxmox VE node fails, virtual machines running on that node stay offline until an administrator restarts them manually. Proxmox VE high availability (HA) solves this by monitoring cluster nodes and automatically restarting protected VMs on a healthy node. The useful takeaway is that HA protects against node and hardware failure, not against crashes inside the guest operating system. It requires a clustered Proxmox environment, shared storage, and fencing.

Mechanism: Cluster, Quorum, and HA Resources

Proxmox VE HA is built on a cluster stack. Each node runs the cluster manager, and nodes vote to maintain quorum. If quorum is lost, HA stops starting new VMs to prevent split-brain, where two nodes try to run the same VM simultaneously. The HA resource manager tracks which VMs are protected and their desired state, such as started or stopped.

For a VM to fail over, its disk image must live on storage that every candidate node can mount. This means shared storage such as Ceph, NFS, or iSCSI. Local directory or LVM-thin storage on a single node cannot be used for HA because another node cannot access the VM disk after failover.

Fencing is the safety mechanism that isolates a failing node before HA restarts its VMs elsewhere. Without fencing, an unresponsive node might still be writing to shared storage, causing data corruption. Proxmox supports fencing agents for power switches, virtualization platforms, and other hardware.

Worked Example: Protect a VM with HA

Run cluster and HA commands on any Proxmox node shell as root. Replace 100 with your VM ID and node1 with your actual node name.

1. Confirm the cluster is healthy and has quorum:

pvecm status

Expected output includes the node list and a quorum status showing the node is a member. If quorum is lost, HA resources cannot start.

1. Verify shared storage is active on all nodes:

pvesm status

Look for the storage used by your VM disks and confirm it shows as active on each node that might host the VM.

1. Add the VM to HA through the web interface:

1. Go to Datacenter > HA > Add.
2. Select Virtual Machine and choose the target VM.
3. Set State to Started.
4. Set Priority if multiple VMs compete for resources during failover.
5. Click Add.

HA resources are stored in /etc/pve/ha/resources.cfg and managed through the cluster filesystem. Do not edit this file directly while the HA service is running.

1. Check the HA resource state:

ha-manager status

Look for an entry like vm:100 with state started and a node assignment. If the state shows stopped or error, the VM is not currently protected.

Limits and Common Mistakes

HA Does Not Replace Backups or Guest Monitoring

HA restarts the VM on another node, but it does not recover data lost inside the guest. If the application inside the VM crashes or the filesystem corrupts, HA sees the VM as still running and will not restart it. Use Proxmox Backup Server or regular snapshots for data protection.

Shared Storage Is Mandatory

A common mistake is placing VM disks on local LVM-thin storage and expecting HA to move them. Failover cannot occur if the destination node cannot read the disk. Confirm the VM's storage is listed as shared in Datacenter > Storage and is available on all HA nodes.

Cluster Size and Quorum

A three-node cluster is the smallest recommended setup that maintains quorum after losing one node. A two-node cluster loses quorum if one node fails unless you configure a quorum device, such as a third Proxmox node or a QDevice. Without quorum, HA will not start VMs.

Fencing Misconfiguration

If fencing is not configured or the fencing device cannot reach the failed node, HA may refuse to fail over to avoid data corruption. Test fencing independently before relying on HA in production.

Verification: Simulate a Node Failure

Before trusting HA, verify failover in a controlled window:

1. Confirm ha-manager status shows the VM as started on node1.
2. Place node1 in maintenance mode through Node > Maintenance, or reboot the node during a maintenance window.
3. Observe ha-manager status on another node. The VM should transition to the healthy node and reach state started.
4. Start the VM console from the web interface to confirm the guest OS boots correctly.

Risk: Rebooting or fencing a node interrupts all VMs on that node until failover completes. Schedule this test during low-traffic periods.

Practical Takeaway

Proxmox VE HA turns a single-node outage into an automatic recovery event, but only when shared storage, quorum, and fencing are correctly configured. Start by protecting one non-critical VM, verify failover with a controlled node reboot, then expand HA coverage to production workloads.