vSphere Distributed Switch Architecture: Design Minimums, Trust Boundaries, and Operational Validation for vSphere 7.0 U3–8.0 U2

The Problem: Inconsistent Network Fabric Across Clusters

When ESXi hosts in a cluster run different virtual switch configurations, vMotion fails, DRS cannot balance workloads, and NSX-T overlay prerequisites go unmet. The vSphere Distributed Switch (VDS) solves this by centralizing L2/L3 policy, port-state visibility, and traffic shaping under vCenter — but only if the design accounts for version coupling, trust boundaries, and failure domains from day one.

Takeaway: Deploy a single VDS per data center with NIOC v3, MTU 9000 uplinks, and explicit port-blocking defaults. Validate daily with automated health checks, and treat vCenter compromise as a network-reconfiguration risk.

Requirements Driving the Design

• Consistent port-group policy (VLAN, PVLAN, security, shaping) across all hosts in a cluster
• vMotion, DRS, and NSX-T Geneve/VXLAN overlay support — requires MTU ≥ 1600 on all uplinks and vmkernel interfaces
• Centralized lifecycle: VDS version follows vCenter version; upgrade coordination is mandatory
• Telemetry: NetFlow/IPFIX export to a collector without starving the management vmkernel

Smallest Suitable Design

One VDS Per Data Center

Create a single VDS (e.g., vds-mgmt) spanning all clusters in the data center. This avoids cross-VDS port-group sprawl and keeps NSX-T transport zone mapping simple.

Uplink Port Groups: One Per Physical NIC Team

Map each physical NIC pair to an uplink port group (e.g., uplink-pg-01, uplink-pg-02). Choose teaming policy per rack:

• Active/Standby for simplicity; failover order explicit
• LACP (vDS 7.0+) for bandwidth aggregation — requires matching physical switch config

Set MTU 9000 on every uplink port group and the underlying physical switch ports. Enable CDP/LLDP advertisement so esxcli network vswitch dvs vmware lldp shows neighbor chassis/port IDs.

Network I/O Control v3

Enable NIOC v3 on the VDS. Define resource pools for traffic types:

Pool	Shares	Reservation (Mbps)	Limit (Mbps)
Management	High (50)	1000	Unlimited
vMotion	High (50)	2000	Unlimited
VSAN / vVols	High (50)	3000	Unlimited
VM Network	Normal (25)	0	Unlimited
NFS / iSCSI	Low (10)	500	Unlimited

Run Get-VDSwitch -Name vds-mgmt | Get-VDSwitchNetworkResourcePool from a PowerCLI session connected to vCenter (requires VDSwitch.Config privilege) to verify the pools match the design doc.

Port-Blocking Defaults

On every distributed port group, set security policy to reject forged transmits, MAC changes, and promiscuous mode unless a documented exception exists (e.g., nested ESXi, load-balancer VMs). Audit with:

Get-VDPortgroup | Select Name, @{N='ForgedTransmits';E={$_.ExtensionData.Config.DefaultPortConfig.SecurityPolicy.ForgedTransmits}}, @{N='MacChanges';E={$_.ExtensionData.Config.DefaultPortConfig.SecurityPolicy.MacChanges}}, @{N='Promiscuous';E={$_.ExtensionData.Config.DefaultPortConfig.SecurityPolicy.PromiscuousMode}}

Run from PowerCLI with VDPortgroup.Config read privilege. Any true value must have a change-record reference.

Trust and Data Boundaries

Control Plane: vCenter

VDS configuration lives in vCenter's inventory database. A compromised vCenter account with VDSwitch.Config or DVPortgroup.Config privileges can rewrite the entire network fabric — VLANs, teaming, NIOC, port-blocking. Mitigate with:

• Separate AD groups for network admins vs. compute admins
• Privilege propagation audit: Get-VIPermission -Entity (Get-VDSwitch vds-mgmt) | Where {$_.Role -match 'Admin|Config'}
• vCenter backup encryption and offline DR copy

Data Plane: ESXi hostd/vpxa Agents

Each host's hostd and vpxa apply VDS config pushed from vCenter. A compromised ESXi host (root access via SSH/console) can mutate local port-group settings via esxcli network vswitch dvs vmware commands, but changes are overwritten on next vCenter sync unless the host is disconnected. Monitor /var/log/vpxa.log for ReconfigVM_Task and ReconfigureDVPortgroup_Task events originating from the host itself.

Tenant Isolation: VLAN / PVLAN / NSX Segments

VM traffic isolation relies on correct trunk/access port configuration. A misconfigured trunk port group (VLAN 4095 (ALL) without PVLAN mapping) leaks tenant traffic across segments. PVLAN promiscuous/isolated/community ports on VDS must match the physical switch PVLAN config — mismatch causes silent connectivity loss with no VDS-side error.

Physical Switch: Separate Admin Domain

Upland switches are typically managed by a network team. Coordinate:

• LACP mode (active/passive) and system-priority
• STP/BPDU guard on access ports facing ESXi — ESXi does not run STP; BPDU guard prevents loops if a VM bridges
• MTU 9000 end-to-end, including inter-switch links

Operational Validation

Daily Automated Checks

Schedule a cron job on a management jump host (or vCenter scheduled task) running:

#!/bin/bash
# Run as root on each ESXi host via SSH key or host profile
for host in $(cat /etc/vds-hosts.txt); do
  ssh $host 'esxcli network vswitch dvs vmware vxlan health check --vds-name=vds-mgmt'
done

This validates VXLAN/VTEP health, MTU consistency, and control-plane connectivity. Alert on any non-zero exit code.

Weekly Configuration Drift Export

Run vdsinfo -p > /var/log/vds-snapshots/vds-$(date +%F).txt on one host per cluster (requires root). Diff against last week's snapshot to catch unauthorized port-group additions, MTU changes, or NIOC pool edits.

Telemetry Health

NetFlow/IPFIX export from VDS uses the management vmkernel interface. If management network saturates, telemetry gaps appear during congestion — exactly when you need visibility. Reserve 1 Gbps for management in NIOC and alert on collector-side zero-flow intervals > 5 minutes on any uplink.

NIOC Saturation Alert

Configure vCenter alarm on Distributed Switch Network Resource Pool Usage > 80% for any pool. When triggered, review esxtop -n (press n for network view) on affected hosts to identify top talkers.

Failure Modes and Mitigations

Failure	Symptom	Mitigation
vCenter outage	VDS changes freeze; data plane continues	Document current config in Git; run DR drill quarterly
Single uplink failure	Teaming fails over; LACP may not fallback to static automatically	Test esxcli network vswitch dvs vmware lacp get after cable pull; verify failover < 1s with continuous VM ping
MTU mismatch on one host	Jumbo frames black-holed for VMs on that host	Daily vmkping -s 8972 -d <peer-vtep-ip> from each host's VXLAN vmkernel
NIOC misconfiguration	Management vmkernel starved; vCenter loses host connectivity	Reservation ≥ 1 Gbps for management pool; alarm at 80%
Stale port-blocking	MAC spoofing possible on VM network	Weekly audit script above; exception list in CMDB

LACP Fallback Caveat

In ESXi 7.0 U3 through 8.0 U1, LACP fallback to static teaming on uplink failure is not automatic in all builds. Verify behavior for your specific build with:

esxcli network vswitch dvs vmware lacp get --vds-name=vds-mgmt --uplink-portgroup=uplink-pg-01

If Fallback shows false, plan manual intervention runbook: disable LACP on the physical switch port-channel, then run esxcli network vswitch dvs vmware lacp set --vds-name=vds-mgmt --uplink-portgroup=uplink-pg-01 --enable=false on the affected host.

Design-Change Triggers

NSX-T Adoption

Requires VDS 7.0+ with enhanced data path (EDP). If current VDS is 6.6 or 6.7, upgrade vCenter first, then upgrade VDS version via Get-VDSwitch vds-mgmt | Set-VDSwitch -Version 7.0.0 (requires maintenance window; hosts reconnect).

Distributed Services Engine (DPU Offload)

vSphere 8.0 U2 introduces VDS 8.0.0 with DPU offload for compatible SmartNICs (e.g., Pensando DSC, NVIDIA BlueField). Existing VDS 7.0 cannot be upgraded in-place to DPU mode. Create a new VDS 8.0, migrate uplinks and port groups, then decommission the old VDS. Requires compatible SmartNIC firmware and vSphere 8.0 U2 on all hosts.

Multi-vCenter Linked Mode with Shared VDS

Requires Enhanced Linked Mode, same SSO domain, and VDS 7.0+. Shared VDS appears in each vCenter's inventory; changes propagate. Test failure domain: disconnect one vCenter, verify the other can still manage the VDS.

Hardware Refresh: 100 GbE NICs

New uplink speed demands NIOC v3 rebalance (reservations in Mbps, not percentages) and new uplink profiles with updated teaming policies. Schedule alongside host firmware upgrade.

Verification Checklist Before Handoff

1. VDS version: Get-VDSwitch | Select Name, Version matches VMware compatibility matrix for vCenter version.
2. MTU end-to-end: vmkping -s 8972 -d <dest-ip> from each host's VXLAN vmkernel to peer VTEP and to physical gateway.
3. NIOC config: Get-VDSwitch -Name vds-mgmt | Get-VDSwitchNetworkResourcePool shares/reservations match design doc.
4. Port-blocking audit: All port groups show false for forgedTransmits, macChanges, promiscuous unless exception documented.
5. vCenter alarms enabled: Alarms → Definitions → search 'Distributed Switch' → health, MTU, uplink, teaming alarms all enabled with email/SNMP actions.
6. Failure domain test: Pull one uplink cable; verify teaming failover < 1s, zero VM packet loss via continuous ping from VM on host A to VM on host B.

Limitations and Open Questions

• VDS version upgrade is one-way; test in lab before production.
• PVLAN on VDS requires exact physical switch match — no VDS-side validation.
• NetFlow/IPFIX export path shares management vmkernel; no separate telemetry vmkernel supported on VDS.
• LACP fallback behavior varies by ESXi build; verify per KB 2032908 for your release.
• DPU offload requires new VDS creation — no in-place upgrade path from VDS 7.0.

Mark any unverified LACP fallback behavior or PVLAN physical-switch parity for review during the next maintenance window.