Live Migration in OpenStack Nova: Planning Zero‑Downtime VM Moves

Why you might need live migration

Imagine you have to apply a firmware update or replace a failing NIC on a compute host that runs dozens of production instances. Taking the host offline would interrupt those workloads unless you can move the running VMs elsewhere without noticeable downtime. OpenStack Nova’s live migration feature lets you do exactly that: shift a running instance from one compute node to another while keeping it accessible to users.

How live migration works under the hood

Nova delegates the actual move to libvirt/QEMU. The process has two main phases:

1. Pre‑copy – libvirt repeatedly transfers dirty memory pages from the source to the destination over the network. While this runs, the VM continues executing on the source host, so users see no interruption.
2. Stop‑copy – once the remaining dirty pages are small enough to transfer within a configurable downtime window, libvirt pauses the VM, sends the final memory state, resumes the VM on the destination, and releases the source resources.

For the disk image to stay available during both phases, the storage backend must allow concurrent access (e.g., NFS, Ceph RBD, or shared block storage). If shared storage isn’t used, Nova can fall back to block migration, which copies the disk while the VM runs but adds extra I/O overhead.

Prerequisites you must verify

• Shared storage – both source and destination compute nodes must mount the same storage pool with identical permissions.
• Libvirt/QEMU compatibility – the versions on both hosts should be within the same major release series (e.g., libvirt 8.2.0 and QEMU 7.2.0) to ensure the domain XML and migration protocol match.
• CPU feature parity – the destination CPU must support at least the same feature set as the source. Using cpu_mode=host-passthrough in the Nova compute configuration helps, but you still need compatible CPU models; otherwise libvirt aborts with an "unsupported CPU" error.
• Network bandwidth and latency – the pre‑copy phase can consume significant bandwidth; a saturated or high‑latency link may cause the migration to exceed the allowed downtime and be rolled back.

Worked example: migrating a running instance

Assume you have an instance named web01 (UUID 123e4567-e89b-12d3-a456-426614174000) running on compute-01 and you want to move it to compute-02 for maintenance.

# Run on a controller or any node with the nova CLI
nova migrate --live 123e4567-e89b-12d3-a456-426614174000 compute-02

Nova will:

1. Ask the scheduler to confirm compute-02 is a valid host.
2. Instruct libvirt on compute-01 to start the pre‑copy phase.
3. Poll the migration status until the task completes.

You can monitor progress with:

nova show 123e4567-e89b-12d3-a456-426614174000

Look for:

• OS-EXT-SRV-ATTR:host changing from compute-01 to compute-02.
• task_state transitioning to migrating and then to None when finished.

To verify the VM stayed reachable:

# From any host with network access to the instance
ping -c 5 
# or
ssh ubuntu@ 'uptime'

During the stop‑copy phase you should see a brief loss of replies (typically under 2 seconds), after which replies resume.

Check libvirt logs for detailed progress:

# On source host
journalctl -u libvirtd | grep -i migration
# On destination host
journalctl -u libvirtd | grep -i migration

Trade‑offs and limitations

• Network load – the pre‑copy phase can saturate the inter‑host link, especially with large memory instances. Consider scheduling migrations during off‑peak periods or using a dedicated migration network.
• CPU overhead** – both hosts run extra threads to compress, transfer, and decompress memory pages, which can temporarily reduce capacity for other workloads.
• Storage concurrency** – if your backend does not support true concurrent access (e.g., a single‑node LVM volume), you must use block migration, which increases migration time and I/O pressure on the storage.
• CPU feature mismatches** – even with host-passthrough, subtle differences in CPU stepping or microcode can cause migration failures. A safe practice is to homogenize the CPU model across the cluster or enable cpu_mode=custom with a vetted baseline.

Actionable checklist before you start

1. Confirm shared storage is mounted identically on source and destination.
2. Verify libvirt/QEMU versions match (or are within the same major series).
3. Check CPU feature flags with virsh capabilities on both hosts; ensure the destination superset includes the source’s features.
4. Measure available network bandwidth (e.g., iperf3) and latency between hosts.
5. Run the migration with the --block-migrate flag only if shared storage is unavailable, and be prepared for longer downtime.
6. After migration, run nova show and inspect libvirt logs to confirm success.

When these conditions are met, live migration provides a reliable way to perform hardware maintenance, firmware upgrades, or load‑balancing moves with virtually no service interruption.