Executing Zero-Downtime VM Live Migration in Proxmox VE

The Challenge of State Transfer

Moving a running Virtual Machine (VM) between physical hosts usually requires a shutdown, causing service interruption. Live migration solves this by transferring the VM's active memory and device state over the network while the VM continues to execute on the source host. The primary technical hurdle is ensuring the destination host can resume the CPU instructions and access the disk exactly where the source left off.

Infrastructure Prerequisites

Live migration requires a synchronized environment. If the destination host lacks a CPU feature the VM is currently using, the process will crash the guest.

1. CPU Compatibility

The source and destination CPUs must be compatible. The safest approach is to set the VM CPU type to host if the hardware is identical, or x86-64-v2-AES (or similar baseline) if the hardware differs. To verify compatibility, run this command on both hosts as root:

lscpu | grep "Flags"

Compare the output. If the source host has flags (like avx2) that the destination lacks, the migration will fail during the memory handoff.

2. Shared Storage Backend

The VM disk must reside on storage accessible by both hosts. Proxmox supports several backends for this:

• Ceph: Distributed block storage providing high availability.
• NFS: A centralized network file system.
• ZFS over iSCSI: Block-level access with ZFS snapshots.

Ensure the storage is defined in Datacenter > Storage and marked as "Shared" in the Proxmox GUI.

3. Network Connectivity

Migration traffic can saturate a management network. It is recommended to use a dedicated migration network. Ensure the Proxmox firewall allows traffic on the required ports (primarily TCP 8006 for API coordination and the specific migration ports defined in the cluster settings).

Migration Procedure

Assuming you are using Proxmox VE 7.x or 8.x, follow these steps to move a VM (VMID 100) from Node A to Node B.

Step 1: Initiate Migration

In the Proxmox Web UI, right-click the VM and select Migrate. Alternatively, use the CLI on the source node:

# qm migrate 100 node-b --online

The --online flag ensures the VM remains running. If the disk is not on shared storage, Proxmox will attempt a "storage migration," which is significantly slower and more resource-intensive.

Step 2: Monitor the Transfer

While the migration is running, you can track the progress on the destination node by checking the task log:

# tail -f /var/log/pve/tasks.log

Look for the migrate entry. The system will iteratively copy memory pages from the source to the destination. In the final phase, the VM is paused for a few milliseconds on the source and resumed on the destination.

Verification and Diagnostics

Once the UI indicates the migration is complete, verify the VM state on the destination host (Node B) using the following checks:

Check	Command (Run on Destination)	Expected Result
VM Status	qm status 100	status: running
Resource Match	qm config 100	Memory and CPU cores match source config
Storage I/O	iostat -x 1 5	Active read/write operations on shared storage volume

Handling Stalled Migrations

If the migration hangs at 99%, it is often due to a "dirty page" problem—where the VM is writing to memory faster than the network can transfer it. You can attempt to force the handoff via the QEMU monitor:

# qm monitor 100
# migrate_set_parameter max-downtime 500

Increasing the max-downtime allows the hypervisor to pause the VM longer to complete the final transfer.

Rollback and Recovery

Since live migration changes the execution host but not the disk data, the primary risk is a crashed VM state on the destination. If the VM fails to resume:

1. Stop the VM on the destination node immediately to prevent disk corruption.
2. Verify Disk Locks: If the VM won't start on the source, check for stale locks on the shared storage (e.g., qm unlock 100).
3. Restore from Snapshot: If the guest OS becomes corrupted during the handoff, use the ZFS or Ceph snapshot taken prior to migration to revert the disk state.