Live Migrating VMs in Harvester with Virtctl: A Practical Guide

Problem

You need to move a running virtual machine from one Harvester node to another without interrupting user‑visible traffic. The move must be quick enough to avoid noticeable downtime, but you also want to understand the requirements and limits before trying it in production.

Thesis

Harvester’s live migration, driven by KubeVirt and triggered with virtctl migrate, can relocate a VM with sub‑second downtime when shared storage and compatible CPUs are present. By following a concrete example and verifying key indicators, you can confidently use this feature for workload balancing or maintenance.

How Live Migration Works in Harvester

When you issue a migration request, Harvester tells KubeVirt to copy the VM’s memory pages from the source node to the target node while tracking writes (dirty pages). Once the amount of remaining dirty memory falls below a threshold, the VM is paused briefly, the final state is transferred, and the VM resumes on the target host. Throughout this process the VM’s disks stay on shared storage (e.g., Longhorn or NFS), so no data movement is required for the disks.

Prerequisites

• A Harvester cluster with at least two nodes.Shared storage accessible from both nodes (Longhorn is the default; external NFS also works).Compatible CPU models on all nodes (check with virsh capabilities or ensure the cluster uses a baseline CPU model).The VM must not use device passthrough (GPU, SR‑IOV, etc.) – live migration will fail for those devices.Sufficient network bandwidth between nodes; 10 GbE is recommended for sub‑30‑second moves of an 8 GB RAM VM.

Worked Example: Migrating a Test VM

Assume you have a VM named web‑01 running on harvester-node-1 and you want to move it to harvester-node-2. You have cluster‑admin rights or a user with the kubevirt.io:virtualmachineinstance verb.

1. Verify the VM’s current node:

   kubectl get vmi web-01 -o jsonpath='{.spec.nodeName}'
   

   Expected output: harvester-node-1

2. Start the migration, waiting for completion:

   virtctl migrate web-01 --wait --timeout 180s
   

   This command contacts the KubeVirt API, selects harvester-node-2 as the target (if only one other node is schedulable), and blocks until the VM reports Running on the new node or the timeout expires.

3. Check the VM’s new node:

   kubectl get vmi web-01 -o jsonpath='{.spec.nodeName}'
   

   Expected output: harvester-node-2

4. Observe downtime from an external client (e.g., a laptop on the same network):

   ping -i 0.2 
   

   During the migration you should see at most a few lost packets (typically 1‑2 out of hundreds), confirming the near‑zero downtime claim.

Verifying Migration Progress

If you prefer to watch the progress live, add the --watch flag:

virtctl migrate web-01 --watch

The output shows a percentage that climbs to 100% as dirty memory shrinks. In the Harvester UI, the VM’s card will display a migration banner and move to the target node’s host list upon completion.

Trade‑offs and Limitations

• Storage I/O: While memory is copied over the network, the VM’s disks remain on shared storage. Heavy write workloads can increase latency on the storage backend during the copy phase; monitor Longhorn or NFS metrics if you notice slowdowns.
• Device passthrough: VMs with GPUs, SR‑IOV NICs, or other passthrough devices cannot be live‑migrated; you must shut them down and perform a cold move.
• CPU compatibility: Migrating to a node with a newer CPU feature set may cause a MachineCheck if the VM relies on instructions unavailable on the source. Enforcing a uniform CPU model (e.g., host-passthrough disabled) avoids this.
• Network bandwidth: Lower‑bandwidth links increase migration time linearly; a 1 GbE link may take several minutes for the same 8 GB RAM VM, extending the window of potential performance impact.

When to Use Live Migration

Live migration is ideal for:

• Rolling node upgrades or firmware updates.
• Load balancing when one node approaches resource saturation.
• Evacuating a node for maintenance without service interruption.

For workloads that require strict I/O latency or use passthrough hardware, plan a scheduled shutdown instead.

Actionable Closing

Before attempting migration in production, run the example above on a non‑critical VM. Verify:

1. The VM’s spec.nodeName updates to the target node.
2. External ping shows minimal packet loss.
3. Storage backend metrics stay within normal bounds during the copy phase.

If all checks pass, you can confidently use virtctl migrate to move VMs as part of your operational routine.