Drupal Core Caching ↔ Varnish Reverse Proxy: When Is the First Miss the Bottleneck?
0 reputation · 17 Apr 2023, 14:13 UTC
0 reputation · 17 Apr 2023, 14:13 UTC
Determine whether the first request after a cache miss is the true performance bottleneck when Drupal core caching is paired with an external reverse proxy such as Varnish.
Drupal 9/10 generate cacheability metadata (tags, contexts, max‑age) that both the internal Dynamic Page Cache and Varnish interpret. The first miss triggers a full bootstrap, but the relative cost of PHP execution versus Varnish processing is unclear, especially under high traffic. Profiling tools add overhead, and the decision to automatically purge Varnish on cache‑tag updates remains unresolved.
Drupal core currently does not guarantee automatic invalidation of external proxies when cache tags change, leaving site operators to decide between manual purge scripts and relying on header propagation.
On a cache miss, the bottleneck is almost always Drupal, not Varnish. A miss forces a full Drupal bootstrap — PHP execution, module loading, database queries — while Varnish contributes only a hash lookup and a backend connection, typically single-digit milliseconds. The practical question is not whether the first miss is expensive (it is, by design) but how to measure the split precisely and how to keep the warm-cache hit rate high enough that misses are rare.
Confirmed behavior (Drupal 9/10, well established): Drupal emits cacheability metadata — Cache-Control, X-Drupal-Cache-Tags, and cache contexts — on cacheable responses. Varnish can honor Cache-Control: max-age directly and can be configured to ban on tag-like surrogate keys. Drupal core does not itself send PURGE/BAN requests to Varnish when cache tags are invalidated; that gap is real and is the source of your unresolved decision.
Likely explanation, not confirmed for your site: that PHP bootstrap dominates miss latency. This is true in the general case, but your actual split depends on modules, database latency, and whether the miss also misses the Dynamic Page Cache and render cache. Measure before tuning.
Avoid XHProf/Xdebug for the baseline — their overhead inflates bootstrap time. Instead:
varnishlog, watching BereqHeader/BerespBody timestamps (or Timestamp records) for a forced miss. The backend fetch duration approximates Drupal's total response generation time.X-Drupal-Cache / X-Generator headers and use the web server's request-time log or a simple curl -w timing against the origin directly, bypassing Varnish.Drupal core does not push invalidations to Varnish. To make tag-based purging work at all you need a contrib module (the Purge module suite with a Varnish purger is the standard approach) plus VCL that accepts BAN requests and matches against the tag header Drupal sends. Verify end to end:
curl -sI https://example.com/node/1 | grep -i -E 'cache-tags|surrogate'.varnishlog for an incoming BAN and a matching ban hit on the cached object.X-Varnish/Age shows a fresh fetch, not a stale hit.If no BAN arrives in the log, the problem is on the Drupal/purger side; if the BAN arrives but nothing is evicted, the VCL ban expression doesn't match the header format.
For most sites: automatic, tag-based invalidation via the Purge module, with manual varnishadm ban kept as an operational fallback. Manual-only purging doesn't scale with editorial activity and guarantees stale content between purges; relying on max-age alone trades correctness for simplicity. The caveat: tag-based bans in Varnish can accumulate and slow lookups if bans are never garbage-collected, so use ban-lurker-friendly VCL (ban on obj.http rather than req.http) and monitor ban list length.
One missing detail changes this recommendation: your Varnish version. Ban behavior and VCL syntax differ meaningfully between Varnish 6.x and 7.x, and the purger configuration must match — confirm which you run before writing the VCL.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.