When One Nova Host Is Too Much: Anti-Affinity Server Groups

A three-node web tier can still be a single point of failure if Nova schedules all three instances onto the same compute host. Nova's default filters consider resource capacity, image and network compatibility, and availability zones; they do not infer that three instances belong to one application tier. The useful takeaway: express the separation requirement as a Nova anti-affinity server group, then verify both the placement and the capacity cost of that requirement.

What the anti-affinity policy actually enforces

A server group is a Nova object that holds a policy and a membership list. When you launch an instance with the group's UUID as a scheduler hint, Nova associates the instance with that group. The scheduler then evaluates candidate compute hosts against the group's policy.

With the anti-affinity policy, group members must not share a compute host. This is a hard constraint: if no candidate host satisfies it, the request fails instead of silently co-locating the instance. Nova also offers soft-affinity and soft-anti-affinity policies, which influence weighted scheduling rather than excluding candidates outright. Use the hard policy when co-location is worse than a failed boot; consider the soft policy when spreading instances is desirable but not mandatory.

Server groups do not retroactively move running instances. Adding a member later does not reschedule existing members, and migrations remain subject to the scheduler's normal policy checks. The group expresses a placement rule for instances scheduled with the hint.

Worked example: create, launch, verify

Run these commands from a machine with the OpenStack client installed and credentials loaded for the intended project. Creating and managing server groups is commonly restricted to administrators under Nova's default policy, while launching instances uses normal project permissions. Confirm the local policy and client syntax before automating the sequence, because option names can differ across releases and distributions.

# Create the group and capture its UUID
openstack server group create --policy anti-affinity web-tier

# Launch two members with the same group hint
openstack server create \
  --flavor m1.small \
  --image ubuntu-22.04 \
  --network private \
  --hint group=GROUP_UUID \
  web-node-1

openstack server create \
  --flavor m1.small \
  --image ubuntu-22.04 \
  --network private \
  --hint group=GROUP_UUID \
  web-node-2

Replace GROUP_UUID with the ID returned by the create command, and replace the flavor, image, and network with values available in your project. The --hint group=... argument is the important part: it associates each new server with the group.

After both servers reach ACTIVE, compare their compute-host assignments:

openstack server show web-node-1 -c OS-EXT-SRV-ATTR:host -f value
openstack server show web-node-2 -c OS-EXT-SRV-ATTR:host -f value

The two values should identify different compute hosts. If they match, check that the hint was included in both create requests and that the UUID refers to the anti-affinity group you created, for example with openstack server group show GROUP_UUID. Do not assume a hard anti-affinity filter was silently ignored. Some deployments also expose the hypervisor name as OS-EXT-SRV-ATTR:hypervisor-hostname; compare that field when the compute host name is not the failure-domain label you need.

Capacity is the trade-off

Anti-affinity consumes placement capacity. In a cluster with three eligible compute hosts, a four-member anti-affinity group has no compliant placement for the fourth member, even if RAM and vCPU capacity appear plentiful. The request should fail with a scheduling error rather than violate the policy; the exact error text depends on the release and the surrounding filters.

Combined constraints make the ceiling harder to see. An availability zone, host aggregate, or additional scheduler filter can reduce the eligible set before anti-affinity is evaluated. A group that works across the whole cluster may fail inside a two-host aggregate. Diagnose the candidate set rather than treating a no valid host message as proof that the cluster lacks raw capacity.

When a boot fails, the practical options are to add eligible compute capacity, reduce the number of instances that must be separated, or switch to soft-anti-affinity if occasional co-location is acceptable. Do not remove the group from deployments reflexively: a failed boot is visible, while a collapsed failure domain may not be.

What anti-affinity does not protect

Host separation addresses compute-host failure only. It does not contain failures in shared storage, a shared network path, power distribution, or an entire rack. Pair it with failure-domain-aware networking and storage design, and test the result rather than treating the group as a complete high-availability strategy.

A small test that proves the behavior

Before rolling the pattern out, run a controlled test in a non-production project: create a group, launch two instances with the hint, and confirm different host assignments. Then attempt to launch more members than there are eligible hosts and record the exact failure message. Finally, inspect Nova scheduler logs during that test to confirm the server-group filter is evaluating candidates. Log paths and placement API details vary by deployment, so use your operator runbook to locate them.

Anti-affinity is a small configuration with a real operational consequence: it trades guaranteed separation for reduced placement flexibility. Size the group against the failure domain you actually need to protect, and keep enough eligible hosts to satisfy it during maintenance and failure.