Configuring and Validating VM Live Migration in Harvester

Node maintenance on a hyperconverged cluster usually means one painful question: which VMs have to go down? Harvester, because it runs VMs through KubeVirt on Kubernetes, supports live migration — moving a running VM's memory state to another node while keeping its network identity and disk attachments intact. Done right, a kernel patch on a host costs you zero guest downtime. Done blindly, it stalls at 99% and you learn about migration timeouts the hard way. This guide covers the prerequisites, the procedure, and how to prove it actually worked.

Behavior and UI placement vary between Harvester releases, so confirm details against the documentation for your deployed version before relying on any specific default value mentioned here.

What live migration requires

Live migration copies a VM's memory pages from a source node to a target node while the guest keeps running, then briefly pauses the guest to transfer the final dirty pages and CPU state. For that to work, several conditions must hold:

• Shared storage. The VM's disks must be visible from both nodes. In Harvester this means Longhorn-backed volumes. Disks on local or hostPath storage pin the VM to its node and block migration.
• No host-bound devices. VMs using PCI passthrough (GPUs, NICs), USB devices, or strict CPU pinning generally cannot migrate, because the target node cannot provide the identical device.
• Spare capacity. The target node needs enough free CPU and memory to hold the incoming VM on top of its existing load.
• A healthy multi-node cluster. Migration is meaningless on a single node, and degraded Longhorn replicas or an unhealthy node will stall or fail the operation.

Before planning maintenance around migration, inventory your VMs for passthrough devices and local disks. Anything non-migratable needs a scheduled shutdown window instead.

Migration limits worth knowing

KubeVirt throttles migrations cluster-wide: there are caps on how many migrations run in parallel per cluster and per node, plus bandwidth limits and a completion timeout. A VM with a large memory footprint or a workload that dirties pages faster than they can be copied (a busy database, for example) can hit the timeout and fail. These knobs are version-sensitive, so check the KubeVirt configuration in your release rather than assuming defaults. If you routinely migrate large VMs, expect to tune bandwidth and timeout values — and test before maintenance day.

Triggering a migration

The simplest path is the Harvester UI: open the VM, choose Migrate, and pick a target node (or let the scheduler choose). The CLI equivalent uses virtctl, run from a machine with kubectl access to the Harvester cluster and cluster-admin or equivalent VM permissions:

virtctl migrate <vm-name> -n <namespace>

This creates a VirtualMachineInstanceMigration object (abbreviated vmim) that tracks the operation. Watch its progress:

kubectl get vmim -n <namespace> -w

For whole-node maintenance, you do not need to migrate VMs one at a time. Draining the node triggers eviction of migratable VMs automatically:

kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data

Run this from your management workstation against the Harvester cluster. The risk: drain will block on non-migratable VMs, so check for those first with kubectl get vmi -A -o wide and confirm which VMs are pinned to the node.

Proving it worked

A green checkmark in the UI is not evidence. Validate with three checks:

1. Migration object succeeded. kubectl get vmim -n <namespace> should show the phase Succeeded. Anything stuck in Running or Failed needs attention.
2. The VMI moved. kubectl get vmi <vm-name> -n <namespace> -o wide should report the target node, and kubectl describe vmi shows the migration event history.
3. The guest survived. The strongest test is end-to-end: before migrating, start a continuous ping or an iperf3 stream from inside (or to) the guest. During a healthy migration you should see zero or a few lost packets during the final switchover, not a connection reset. TCP sessions should persist.

Also confirm storage stayed healthy throughout:

kubectl get volumes -n longhorn-system

Volumes should remain in a healthy, attached state before, during, and after migration. A volume flapping between nodes or entering a degraded state during migration is a signal to stop and investigate before draining anything else.

When migration stalls or fails

If a migration hangs — common with large-memory or write-heavy VMs hitting the timeout — you can abort it without harming the guest. Deleting the migration object cancels the operation and leaves the VM running on the source node:

kubectl delete vmim <migration-name> -n <namespace>

After aborting, check three things before retrying: whether the VM's memory dirty rate exceeds the migration bandwidth (a busy workload may simply need a quieter window), whether the target node has genuine headroom (kubectl top node gives a rough picture), and whether the configured timeout and bandwidth limits are appropriate for your VM sizes in your Harvester release. Retrying the same operation against the same constraints usually produces the same failure.

A safe rehearsal

Do not make your first live migration a production database during an emergency patch window. Create a small test VM on a Longhorn-backed storage class, start a continuous ping to it, migrate it between two nodes, and count packet loss. Then drain a node in a maintenance window and watch all migratable VMs evacuate while non-migratable ones are flagged. That rehearsal tells you your real migration duration, your real dirty-page behavior, and which of your VMs were never migratable in the first place — all before it matters.