Live Migrate OpenStack Nova Instances Using Shared Storage

The problem: local disk means no live migration

You need to patch a compute host, but an instance on that host uses local disk. OpenStack Nova cannot live-migrate the instance because the destination host cannot read or write the instance's disk image. Your choices become a cold migration with downtime, or leaving the instance where it is.

The useful takeaway is straightforward: when every compute node can access the same disk image through shared storage, Nova can copy memory and CPU state while the VM keeps running. The disk stays on the network storage, so the destination host can start using it immediately after the state transfer.

Why shared storage changes the migration path

Nova coordinates the move, but libvirt and QEMU do the actual work. During live migration, QEMU sends memory pages to the destination host while the instance keeps running. At the end, it pauses the VM briefly, sends the remaining dirty pages, and resumes execution on the destination. This works only if the destination can open the instance's disk. With local disk, it cannot. With NFS, CephFS, or GlusterFS mounted at the same path on both hosts, it can.

Prerequisites before you migrate

• A shared storage export mounted at an identical path on every compute node, for example /var/lib/nova/instances.
• The same nova system user and group ID on all compute nodes so file ownership matches.
• NFS export options that allow root access from compute nodes, such as no_root_squash, or equivalent permissions for your storage protocol.
• Compatible libvirt and QEMU versions on source and destination.
• Admin credentials for the OpenStack client, because live migration is a privileged operation.

Worked example: two-node NFS deployment

Assume compute nodes named compute-01 and compute-02, and an NFS server exporting /export/nova-instances. Mount it on both compute nodes at /var/lib/nova/instances.

Add an /etc/fstab line on each compute node:

nfs-server:/export/nova-instances  /var/lib/nova/instances  nfs  defaults,no_root_squash,async  0  0

Run mount -a on both nodes, then confirm the mount appears identically:

mount | grep /var/lib/nova/instances

In Nova's libvirt configuration, enable live migration flags. The exact key depends on your OpenStack version, but a common setting is:

live_migration_flag = VIR_MIGRATE_LIVE

From a host with admin OpenStack credentials, verify both compute services are up:

openstack compute service list

Then issue the live migration. Run this from your controller or workstation, not inside the instance:

openstack server migrate --live compute-02 <instance-id>

Replace <instance-id> with the UUID of the running instance. Required permissions: an account with the admin role or equivalent compute rights. Expected check: the instance status moves to MIGRATING and returns to ACTIVE on compute-02. Relevant risk: if the NFS path is slow or unreachable, the migration can time out; the instance usually remains on the source host, but verify before proceeding with host maintenance.

Verify the result and know the limits

Check migration status from the controller:

openstack server migration list

While migration runs, confirm the instance is still reachable:

ping <instance-ip>

After completion, confirm the disk path is on shared storage on the destination node:

mount | grep /var/lib/nova/instances

Libvirt logs on both hosts, typically under /var/log/libvirt/qemu/, show migration progress and completion messages.

Trade-offs and limitations

Factor	Local disk	Shared storage
Live migration possible	No	Yes
Typical downtime	Instance shutdown required	Sub-second pause at cutover
Storage failure impact	One host affected	All instances using the share lose disk access
Network dependency	Migration network only	Storage network plus migration network

Shared storage adds latency and becomes a single point of failure for every instance on that export. If the NFS server goes down, running instances lose access to their disks and may crash. Live migration also does not work for every workload; instances with local ephemeral disks or direct hardware passthrough typically cannot be moved this way.

Practical closing step

Test with a non-production instance first. Confirm both compute nodes mount the same export with the same options and ownership, then run a short migration while pinging the instance. If the migration stalls, check openstack server migration list and libvirt logs before scheduling maintenance on the source host.