OpenStack Live Migration: Choosing the Right Strategy for Zero-Downtime Maintenance

The Real Problem: Patching Hosts Without Angry Tenants

If you run an OpenStack cloud, compute nodes will eventually need kernel patches, firmware updates, or hardware swaps. The naive approach—shut down instances, patch, reboot—means tenant-visible downtime. Nova live migration is the mechanism that lets you evacuate a host while instances keep running. But "enable live migration" is not a single decision. It's a set of engineering choices about migration mode, storage layout, CPU compatibility, and network capacity, and the wrong combination turns a routine maintenance window into a stuck-migration incident.

The useful takeaway: live migration works reliably when you match the migration strategy to your workload and infrastructure, and you verify convergence with a real test before trusting it in production.

Pre-Copy vs Post-Copy: The Convergence Problem

Traditional live migration uses pre-copy: the hypervisor iteratively copies dirty memory pages to the destination while the VM keeps running, then pauses the VM briefly for a final sync and handover. The tenant-visible downtime is typically a sub-second pause—often undetectable.

The failure mode is convergence. A memory-intensive workload (a busy database, an in-memory cache) can dirty pages faster than the network transfers them. Migration then loops indefinitely or gets aborted. Two mitigations exist: auto-convergence, which throttles the VM's CPU to slow dirtying, and post-copy, which switches the VM to the destination early and faults remaining pages in over the network on demand. Post-copy guarantees completion but degrades performance while pages are fetched—and if the destination host dies mid-migration, the instance is unrecoverable. These features matured around the Liberty–Newton era; confirm availability and exact config keys against the release you actually run.

Quick Comparison

Storage Decides Your Migration Speed

True live migration assumes shared storage—instance disks on NFS or Ceph RBD—so only memory crosses the wire. If your instances boot from local disks, you need block migration, which copies the entire disk to the destination. A 100 GB local disk over a congested network can stretch a migration from seconds to tens of minutes, and the destination must have enough free local space to receive the copy.

The practical decision: if non-disruptive maintenance is a priority, shared storage for instance disks is the single biggest enabler. If you're committed to local disks for performance or cost reasons, budget for longer evacuation windows and test block migration duration with realistic disk sizes before you depend on it.

Configuration Example

On each compute node, in /etc/nova/nova.conf (requires root or config-management access; restart the nova-compute service to apply):

[libvirt]
# Common baseline CPU model so hosts are migration-compatible
cpu_mode = custom
cpu_models = Haswell
live_migration_scheme = tcp

[DEFAULT]
# Cap migration bandwidth (MiB/s) to protect tenant traffic
live_migration_bandwidth = 200

Post-copy is triggered per-migration rather than purely via config in most releases: openstack server migrate --live <server> with post-copy requested via the appropriate microversion or nova live-migration --post-copy on older clients. Check your release's Nova docs for the exact invocation—this changed across versions.

CPU Compatibility and Network Isolation

Migration fails when source and destination CPUs expose different feature sets. Pinning a common named model (like Haswell above) trades access to newer CPU instructions for migration freedom. Alternatively, group homogeneous hardware into host aggregates so the scheduler only places migratable instances on compatible hosts. Verify compatibility before relying on it: compare virsh capabilities output or openstack hypervisor show <id> CPU details across hosts.

Migration traffic is bulk data transfer and will saturate a shared management network. Standard practice is a dedicated migration interface or VLAN plus the bandwidth cap shown above. Without isolation, a host evacuation can degrade every tenant on that network segment.

Trade-Offs and Limitations

• Not HA. Live migration requires a healthy source host. If a node fails unexpectedly, its instances die with it—use instance HA or rebuild semantics for failure scenarios.
• PCI passthrough. Instances with SR-IOV ports or GPU passthrough historically cannot be live migrated; support is partial and version-dependent. Verify before promising it.
• Stuck migrations. A failed migration can leave an instance in a transitional state requiring manual confirmation or revert (openstack server migration commands or older nova resize-confirm/revert flows).

Verify Before You Trust It

Run openstack server migrate --live <test-server> against a test instance while pinging it continuously, and count the dropped or delayed ICMP replies—that's your real downtime figure. Then repeat with a memory-stressed instance to confirm pre-copy converges, or decide you need post-copy. If you use local disks, time a block migration with a representative disk size. These three measurements tell you more than any configuration guide, and they're the difference between a maintenance window and an outage.