Live Migrating OpenStack VMs Without Downtime: A Practical Guide

Why you might need live migration

When a compute node requires hardware maintenance, a firmware upgrade, or you want to rebalance workloads, moving a running virtual machine (VM) to another host without noticeable interruption is valuable. OpenStack’s live migration feature lets you do exactly that, provided the environment is set up correctly.

Thesis

OpenStack can move a running instance between compute hosts with only a few seconds of downtime when shared storage (or block migration) is available, SSH‑based libvirt authentication is configured, and the hypervisor supports the operation.

Prerequisites and configuration

Before issuing a migration command, verify the following on every compute node:

1. Shared storage – The instances’ disks must reside on a filesystem accessible by both source and destination hosts (e.g., NFS, Ceph, or a shared SAN). If you intend to use block migration instead, ensure the backend supports it and that live_migration_block_migration is enabled in nova.conf.
2. Libvirt and QEMU/KVM versions – Libvirt ≥ 1.0 and QEMU/KVM that support live migration (most recent distributions do).
3. SSH key‑based authentication – The nova user on each host must be able to SSH to the other host without a password. Typically you place the public key of the source host in /var/lib/nova/.ssh/authorized_keys on the destination and vice‑versa, then test with ssh nova@dest-host true.
4. Nova configuration – In /etc/nova/nova.conf on both hosts set:

   [libvirt]
   live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,LIVE_MIGRATE_PEER2PEER
   live_migration_uri=qemu+ssh://nova@%s/system
   

   Restart the nova-compute after changes.

How live migration works

The process consists of three phases:

1. Pre‑phase – While the VM runs, libvirt copies clean memory pages over the network to the destination host.
2. Stop‑copy phase – The VM is briefly paused; any remaining dirty pages (those changed during the pre‑phase) are transferred. This pause usually lasts only a few seconds.
3. Resume – The VM is resumed on the destination host, and the source host releases its resources.

If shared storage is used, the disk image does not need to be copied; only memory state travels over the network. With block migration, the disk is streamed in parallel, increasing bandwidth usage.

Worked example: migrating an instance

Assume you have an instance named web01 with ID 1234abcd-5678-90ef-ghij-klmnopqrstuv currently running on compute-01. You want to move it to compute-02.

1. Check current placement

   openstack server show web01 -c OS-EXT-SRV-ATTR:host -f value
   

2. Initiate live migration (run as a user with the compute:server:migrate permission, typically admin or a role that includes it):

   openstack server migrate --live web01 compute-02
   

3. Monitor progress

   openstack server show web01 -c status -c OS-EXT-SRV-ATTR:host -f value
   

   You will see the status change from MIGRATING to ACTIVE and the host field update to compute-02.
4. Verify success
   • Check nova-compute logs on both hosts for a line containing Live migration successful.
   • Libvirt logs (/var/log/libvirt/libvirtd.log) should show migration completed.
   • From a client, ping the instance’s IP; you should observe a brief spike (sub‑second) then normal latency.
   • If you have access to the VM’s console (e.g., via VNC or SPICE), confirm it remains reachable throughout.

Trade‑offs and limitations

Live migration is not free of costs:

• Network bandwidth – Memory pages (and possibly disk blocks) are streamed; a busy migration can saturate a 1 GbE link, affecting other tenant traffic.
• Source host load** – While copying pages, the source CPU experiences extra overhead due to memory scanning and network I/O.
• Incompatible workloads** – Instances using device passthrough (GPU, SR‑IOV VF, USB devices) or hardware‑specific features cannot be moved because the state is tied to the physical device.
• Local ephemeral disks** – If an instance’s root disk is local to the compute node and not on shared storage, migration will fail unless block migration is explicitly enabled and the backend supports it; otherwise data loss occurs.

Before migrating a production workload, verify that the instance’s flavor does not request passthrough devices and that its disks are on a shared store or that block migration is tested in a staging environment.

Actionable closing

To safely adopt live migration in your OpenStack cloud:

1. Audit all instances for shared storage or block‑migration readiness.
2. Configure SSH keys and nova.conf as described, then restart nova-compute on every compute node.
3. Run a test migration on a non‑critical VM, monitor logs, and confirm ACTIVE status on the destination host.
4. Document the expected downtime window (usually < 5 s) and communicate it to stakeholders.
5. Add a migration step to your runbook for host maintenance, and include a rollback plan: if migration fails, the instance remains on the source host; you can simply retry after fixing the underlying issue (e.g., network or storage).

When the prerequisites are met, live migration becomes a reliable tool for zero‑downtime hardware maintenance and workload balancing.