Practical xDS Decisions: When ADS, Delta, and Resource Warming Actually Matter

The Real Problem: Config Churn at Scale

You're running Envoy at the edge or in a service mesh. Restarting proxies for every route change is off the table. The control plane pushes updates, but something feels off — memory spikes during large EDS pushes, occasional 503s during config rolls, and the control plane CPU climbs when you add the hundredth proxy. The xDS APIs solve dynamic configuration, but the defaults don't scale automatically.

Thesis: Match the xDS Pattern to Your Fleet Size and Churn Rate

Envoy's xDS family (LDS, CDS, EDS, RDS, SDS) separates control plane from data plane. ADS multiplexes them over one gRPC stream. Delta xDS sends incremental updates. Resource warming pre-validates new config before draining old. Each adds complexity. The right choice depends on proxy count, update frequency, and failure tolerance — not on what the docs recommend as "best practice."

ADS vs. Individual xDS Streams

ADS (Aggregated Discovery Service) bundles LDS, CDS, EDS, RDS, and SDS over a single bidirectional gRPC connection. For fleets under ~500 proxies with moderate churn (tens of updates per minute), individual xDS streams are simpler to debug — each stream has its own backoff, logging, and NACK/ACK semantics. ADS shines when you need atomic snapshot semantics: a route change and its corresponding cluster update apply together, avoiding the window where a new route points to a not-yet-existent cluster. If your control plane already versions resources consistently, ADS reduces connection overhead and eliminates cross-stream race conditions.

Practical check: inspect /config_dump on the admin endpoint. If you see version_info mismatches across resource types during rolls, ADS with a single version token per snapshot eliminates that class of bug.

Delta xDS: The Threshold Where It Pays Off

Delta xDS (supported since Envoy 1.14) sends only changed resources instead of full state. The research flags >100k endpoints as a memory pressure point. But the real trigger is control-plane CPU during full-state resync after a reconnect. With 1,000 proxies each holding 500 endpoints, a control-plane restart triggers 500,000 endpoint objects serialized and sent simultaneously — the thundering herd problem.

Delta xDS adds complexity: the control plane must track per-proxy resource versions and handle subscribe/unsubscribe semantics. If your fleet is under 500 proxies and EDS payloads stay under 50k endpoints, full-state xDS with proper backoff (exponential, jittered, max 30s) is easier to operate. Cross the 1,000-proxy or 100k-endpoint line, and Delta becomes a net win.

Worked Example: Resource Warming for Zero-Downtime Cluster Rolls

Scenario: you're shifting traffic from service-v1 to service-v2 by updating the EDS endpoint list. Without warming, Envoy can drain the old cluster before the new one passes health checks — causing a brief 503 window.

# Cluster config snippet showing warmup
clusters:
- name: service-v2
  type: EDS
  eds_cluster_config:
    service_name: service-v2
  health_checks:
  - timeout: 5s
    interval: 10s
    unhealthy_threshold: 2
    healthy_threshold: 3
    http_health_check:
      path: /health
  # Wait 30s after health checks pass before accepting traffic
  warmup_duration_sec: 30

How it works: when the control plane pushes the new EDS endpoints for service-v2, Envoy starts health-checking them. Only after healthy_threshold consecutive passes plus warmup_duration_sec does Envoy consider the cluster ready. The old cluster drains concurrently. Result: zero 503s if warmup_duration_sec exceeds your health-check interval times threshold.

Verification: hit the admin endpoint /clusters?format=json during a roll. Look for health check state transitioning to healthy and warmup state progressing before traffic shifts. If the old cluster drains before the new one is ready, increase warmup_duration_sec.

Trade-off: SDS and the Control Plane Trust Boundary

SDS rotates TLS certs without proxy restarts — essential for mTLS. But it expands the trust boundary: a compromised control plane can inject malicious certificates, enabling MITM across the mesh. In practice, this means:

• Run the control plane on a dedicated network segment with mutual TLS to Envoy.
• Use SPIFFE/SPIRE or similar for control-plane identity, not static tokens.
• Audit SDS certificate TTLs — short TTLs (1–2h) limit blast radius but increase control-plane load.

If you can't secure the control-plane channel, filesystem-based cert rotation with a short-lived sidecar (for example, cert-manager plus a reload script) is safer than SDS over an untrusted path.

Actionable Closing: Start with Observability, Then Optimize

Before choosing ADS, Delta, or tuning warmup:

1. Enable /config_dump sampling in your monitoring. Track version_info drift across resource types.
2. Measure control-plane CPU and network during a full resync (simulate by restarting the control plane).
3. Load-test with fortio or hey while triggering config updates. Target: zero 5xx and minimal p99 latency increase during rolls.
4. If resync exceeds 30s or control-plane CPU spikes above 80%, adopt Delta xDS. If cross-resource version mismatches appear, adopt ADS. If 503s appear during cluster rolls, tune warmup_duration_sec.

xDS isn't a single feature — it's a toolkit. The engineering decision is which tools your current scale and reliability targets actually justify.