Diagnosing CPU Pinning Problems on an OpenStack Nova Compute Host

The recognizable condition

A compute node shows sustained high CPU load, VMs on that host suffer latency spikes, and when you look at per-core usage with top -H or htop, vCPU threads are spread across every core instead of being confined to the dedicated set you configured. In other words: you asked Nova for CPU pinning, and the host is behaving as if you never did. This guide walks through confirming that, finding the cause, fixing it, and knowing when to escalate.

Version assumption: the configuration names below reflect Nova from roughly the Ussuri/Train era onward, where the legacy cpu_pin_set option was replaced by cpu_dedicated_set and cpu_shared_set. If you run an older release, check your version's nova.conf documentation before applying anything here.

Cause and diagnostic table

Symptom	Likely cause	Where to confirm
VMs boot but float across all cores	Flavor/image never requested pinning (hw:cpu_policy=dedicated missing)	openstack flavor show and image properties
Pinning requested but ignored	cpu_dedicated_set unset or empty in nova.conf	/etc/nova/nova.conf on the compute host
VMs fail to boot with "no valid host" or resource claims fail	Pinning set overlaps reserved/shared cores or is empty after host CPU changes	nova-compute log, placement allocation checks
Config looks right, threads still unbound	Kernel/cgroup problem: CPU controller not available, or libvirt failing to apply cpuset	/sys/fs/cgroup, libvirt logs, VM XML via virsh dumpxml
Latency spikes despite correct pinning	NUMA misalignment or emulator threads sharing pinned cores	numactl --hardware, hw:emulator_threads_policy

Ordered checks

1. Confirm the workload actually asked for pinning. On a controller node (or anywhere with the OpenStack CLI and cloud admin/member credentials), run openstack flavor show <flavor> and look for hw:cpu_policy='dedicated' in the properties. Without this, Nova will never pin, regardless of host config. Check the image too — image properties can override or conflict with flavor extra specs.
2. Check the host's dedicated CPU set. On the compute host, as root or a user who can read Nova config, inspect /etc/nova/nova.conf for cpu_dedicated_set (and cpu_shared_set if you mix pinned and unpinned guests). The range must match real host CPUs — verify with lscpu. A range referencing CPUs that don't exist, or that overlaps the shared set, is a common silent failure.
3. Check what libvirt actually applied. On the compute host, run virsh dumpxml <instance-name> (root or libvirt group membership required) and look for a <cputune> section with <vcpupin> entries. If the XML has no pinning, the problem is in Nova config or scheduling. If the XML has pinning but threads still float, suspect the cgroup layer.
4. Verify the cgroup CPU controller. Inspect /sys/fs/cgroup on the host. On cgroup v2 systems, confirm the cpuset controller is enabled in the subtree control files; on v1 systems, confirm the cpuset/cpu mounts exist. Kernel boot parameters that disable controllers (for example cgroup_no_v1=all on a stack expecting v1) can prevent pinning from being applied. Check the kernel command line with cat /proc/cmdline.
5. Read the logs. In /var/log/nova/nova-compute.log and /var/log/libvirt/libvirtd.log, search for the instance UUID around boot time. Nova logs the CPU set it computed; libvirt logs failures to write cgroup settings. A mismatch between "Nova computed a cpuset" and "libvirt applied nothing" points at the cgroup/kernel layer.
6. Check NUMA placement. Run numactl --hardware and compare against the pinning set. A dedicated set that spans NUMA nodes, combined with a guest that doesn't request a matching NUMA topology, produces exactly the latency-spike symptom even when pinning technically works.

Fixes tied to findings

Flavor missing the policy: set it on the flavor (admin credentials, controller node): openstack flavor set --property hw:cpu_policy=dedicated <flavor>. This only affects new or resized instances — existing VMs keep their old XML until rebuilt or hard-rebooted in a way that regenerates it.

Host config missing or wrong: edit /etc/nova/nova.conf on the compute host and set, for example, cpu_dedicated_set = 8-31 (leaving cores 0–7 for the host OS and shared guests), then restart the service: systemctl restart nova-compute. This is a state-changing operation: restart during a maintenance window, and keep a copy of the previous config so you can restore it and restart again if instances fail to spawn. Risk: if you shrink the set below what running pinned VMs already consume, those VMs are unaffected until reboot, but new placements will fail.

Cgroup/kernel issue: adjust the kernel boot parameters (remove controller-disabling flags, or align the stack on one cgroup version), update the bootloader config, and reboot the host. This requires evacuating or shutting down guests first — plan a full maintenance window and live-migrate or cold-migrate instances off beforehand. Rollback is restoring the previous boot entry.

NUMA misalignment: either narrow cpu_dedicated_set to cores within one NUMA node, or add explicit guest NUMA topology via flavor properties (hw:numa_nodes) so Nova schedules the guest onto a single node. Consider hw:emulator_threads_policy=isolate (with a dedicated core in the shared set) so emulator threads don't steal time from pinned vCPUs.

Verifying the fix

• Boot or hard-reboot a test instance with the dedicated policy, then run virsh dumpxml on the host and confirm <vcpupin> entries exist and reference cores inside your dedicated set.
• Run top -H on the host and confirm the instance's vCPU threads stay on the expected cores under load. A controlled stress test inside the guest (for example stress-ng --cpu 4, run inside the VM, not on the host) should move only the pinned cores.
• Check nova-compute logs for the instance UUID to confirm the computed CPU set matches your intent.

Do not assume the fix worked because the service restarted cleanly — pinning failures are usually silent at the service level and only visible in per-instance XML and thread placement.

Limitations and escalation

Pinning removes the hypervisor's freedom to rebalance vCPUs across cores, so host maintenance and failure handling get harder; treat the dedicated set as a capacity commitment. Exact option names, defaults, and cgroup behavior vary by OpenStack release, distribution, and kernel — verify against your deployed versions rather than assuming this guide's names apply verbatim.

Escalate to your platform or infrastructure team when: the XML and cgroups are correct but latency spikes persist (possible hardware, firmware, or hypervisor issue); NUMA topology on the host doesn't match what Nova reports (possible libvirt/driver bug); or placement shows allocations that disagree with actual host usage (possible resource tracker corruption requiring careful cleanup).