ProcessWire template cache vs ProCache under concurrent requests: which layer should own stampede protection?

The Short Answer

Stampede protection should be owned by the outermost layer. In ProcessWire 3.x, the core template cache does not provide regeneration locks or stale-while-revalidate mechanisms; it is expected that concurrent requests will trigger duplicate renders upon expiry. To prevent this, you must use ProCache (or a reverse proxy), which serves static files directly from the web server, bypassing PHP entirely until the cache is explicitly invalidated.

Behavioral Analysis

Core Template Cache (MarkupCache)

The core template cache is a TTL-driven file system. When a cache file expires, the first request to arrive triggers a re-render. However, because there is no mutex or locking mechanism in the MarkupCache render process, any other concurrent requests arriving before the first process completes the write will also trigger a full render. This is the definition of a cache stampede.

ProCache and Static Layers

ProCache shifts the boundary from PHP to the web server (Apache/Nginx). Because the web server serves a static .html file, no PHP is executed regardless of the number of concurrent requests. The "regeneration" only happens when a page is saved or a cache-clear event is triggered, which then regenerates the static file. This effectively eliminates the stampede risk for the end-user.

Implementation and Coherence

Mixing core template caching and ProCache on the same template is not recommended. Doing so creates a "double-cache" scenario where invalidation becomes unpredictable and increases the risk of serving stale content.

To maintain coherence between the static layer and the CMS:

• Use ProCache for full-page output: Rely on its built-in hooks (Pages::saved, Templates::saved) to purge static files.
• Use WireCache for data fragments: Use the WireCache API for expensive database queries or API calls within your templates. Note that ProCache does not automatically clear WireCache entries; you must hook into Pages::saved to clear specific WireCache keys manually.
• Exclude Dynamic Content: Ensure sessions, forms, and logged-in users are excluded via ProCache settings to avoid serving cached CSRF tokens or private data.

Shared Filesystems (NFS)

If your site is hosted on NFS or a shared filesystem, file-based caching is risky. Core cache writes are not atomic, and flock() is often unreliable on NFS. This can lead to corrupted cache files during concurrent writes. In these environments, you should move the cache directory to a local disk (e.g., /dev/shm or a local SSD) or use a database-backed cache provider.

Verification

To verify if your current setup is susceptible to stampedes, run a concurrency test using Apache Benchmark (ab) immediately after clearing the cache:

ab -n 20 -c 5 https://your-site.com/target-page/

Monitor your PHP process list (e.g., top or htop). If you see 5 simultaneous PHP processes spiking for a single URL, the core cache is the active layer and is not protecting against stampedes.

Diagnostic Note: Are you using a custom WireCache implementation for fragment caching? If so, the recommendation to use a database-backed store for NFS is critical.