Optimizing vMotion for Low-Latency Workloads
To minimize VM stun duration, the most effective configuration is the implementation of Multi-NIC vMotion combined with 10GbE (or faster) networking and Jumbo Frames (MTU 9000). For environments supporting it, utilizing vMotion over RDMA (Remote Direct Memory Access) provides the lowest possible latency and CPU overhead during the memory transfer phase.
Technical Explanation and Assumptions
The "stun" occurs during the final synchronization phase when the VM is paused to transfer the remaining dirty memory pages and device state. This duration is primarily a function of the ratio between the VM's memory change rate (churn) and the available network throughput. If the network cannot evacuate dirty pages faster than the application creates them, the final stun window expands to ensure state consistency.
Assumptions: This guidance assumes a vSphere 7.0+ environment with shared storage. If using storage vMotion (shared-nothing), stun times are typically higher due to the additional disk mirroring phase.
Recommended Configuration Steps
- Enable Multi-NIC vMotion: Assign multiple VMkernel adapters for vMotion to aggregate bandwidth, reducing the time required to clear the memory queue.
- Configure Jumbo Frames: Set the MTU to 9000 across the entire path (VMkernel, physical switches, and NICs) to reduce packet overhead and CPU interrupts.
- Isolate Traffic: Place vMotion traffic on a dedicated VLAN to prevent congestion from management or production data packets.
- Verify Host Resources: Ensure the destination host has sufficient unreserved CPU cycles to resume the VM immediately after the stun phase.
Verification and Diagnostics
To verify the impact of these changes, use esxtop on the source and destination hosts during a migration to monitor network throughput and CPU wait times. You can also review the vSphere Client 'Summary' tab for the specific VM to analyze the migration duration logs.
Addressing Specific Constraints
- Throughput Thresholds: There is no single universal millisecond threshold, as stun duration is relative to the VM's memory write rate. However, 10Gbps is the baseline recommendation for latency-sensitive apps to keep stuns within typical application timeout windows.
- EVC Overhead: Enhanced vMotion Compatibility (EVC) manages CPU feature masking to allow migration between different CPU generations. It does not introduce measurable overhead to the final memory synchronization phase or the stun duration.
Missing Diagnostic Detail: To provide a more precise throughput recommendation, the average memory page change rate (MB/s) of the application during peak load is required.
vsphere