Live Migration in OpenStack: Moving Running VMs Without Downtime

Problem: Needing to maintain compute hosts without interrupting workloads

Operators often need to patch, upgrade, or rebalance compute nodes in an OpenStack cloud. Taking a node offline traditionally means stopping every virtual machine on it, which creates a maintenance window and can affect SLAs. The goal is to move a running VM from one host to another while the VM continues to serve requests, preserving its memory state, network connections, and uptime counter.

Thesis: OpenStack Nova’s live migration feature, built on libvirt/QEMU-KVM, lets you shift a running instance between compatible hosts with minimal visible interruption, provided shared storage or block migration is available and CPU models match.

How Live Migration Works

When you issue a live-migration request, Nova tells the source compute service to:

1. Check that the destination host satisfies CPU compatibility and has enough free resources.
2. Open a TCP channel defined by live_migration_uri between source and destination libvirt daemons.
3. Iteratively copy memory pages while the VM runs, using dirty-page tracking to resend pages that changed during the copy.
4. Transfer the disk image. If the disk resides on shared storage, only metadata is moved; otherwise block migration copies the disk.
5. Pause the VM briefly to sync the final memory state and switch over to the destination host.
6. Resume the VM on the destination and clean up resources on the source.

The process is monitored via the os-migration API; Nova can automatically retry on transient failures such as network glitches.

Worked Example: Preparing and Executing a Live Migration

Assume a two-node OpenStack deployment with shared NFS storage and nova-compute running on both hosts.

1. Verify prerequisites

• Ensure both compute nodes run compatible libvirt/QEMU versions and report compatible CPU models. Check with:

# On each host, run as root or with sudo
virsh capabilities | grep -i cpu

• Confirm live_migration_uri is set identically in /etc/nova/nova.conf on both nodes, e.g.:

live_migration_uri = qemu+ssh://%s/system

• Make sure the nova user has SSH key-based access to the other host without a password prompt.

2. Launch a test instance

# Run as openstack admin
openstack server create --flavor m1.tiny --image cirros-0.5.2 --nic net-id=$(openstack network list -f value -c ID) test-vm

Note the instance ID and the host it is currently on:

openstack server show test-vm -f value -c OS-EXT-SRV-ATTR:host

3. Initiate live migration

Choose the destination host, e.g., compute2. Run:

nova live-migration test-vm compute2

4. Monitor progress

Check instance status:

nova show test-vm

Watch OS-EXT-STS:vm_state transition from active to migrating and back to active. Tail the compute logs:

sudo tail -f /var/log/nova/nova-compute.log

Look for migration progress messages and ensure no errors about CPU incompatibility or storage timeouts.

5. Verify continuity

From an external host, ping or SSH to the instance’s fixed IP throughout migration. The uptime inside the VM should not reset.

Trade-offs and Limitations

• CPU compatibility: If the destination host lacks a CPU feature present on the source, libvirt will refuse migration unless you configure a common CPU model in nova.conf or use host-passthrough with compatibility flags.
• Network and storage bandwidth: Memory dirty-page rate and disk transfer speed directly affect migration time. Block migration of a large qcow2 disk can extend migration time and temporarily increase I/O latency on the source.
• Transient failures: Network blips or storage timeouts can cause migration to abort. Nova’s automatic retry mitigates this, but repeated failures may require manual intervention.
• Shared storage requirement for efficiency: Without shared storage, block migration copies the entire disk, which can be prohibitive for large volumes. Shared storage limits transferred data to memory and metadata.

Actionable Closing

Live migration is a practical tool for reducing maintenance windows and balancing load. To make it reliable:

1. Standardize CPU models across compute nodes or define a common baseline.
2. Deploy shared storage for instance disks whenever possible.
3. Size the migration network and tune live_migration_bandwidth to avoid saturating production traffic.
4. Validate the setup with a test instance before relying on it for production workloads.

By following these steps, you can move running VMs between hosts with minimal user impact during hardware updates or scaling events.