Reducing Headless Browser Overhead with Puppeteer Request Interception

The Cost of Loading Everything

When running Puppeteer for scraping or automated testing, the default behavior is to act like a real user: it downloads every image, executes every tracking script, and fetches every CSS file. In a production environment, this is often a waste of bandwidth and CPU. If your goal is to extract data from a DOM or verify a functional flow, loading a 2MB hero image or a heavy third-party analytics suite provides zero value but adds significant latency.

The solution is Request Interception. By hooking into the browser's network layer, you can decide which requests are essential and which should be dropped before they ever leave the Chromium instance.

How Request Interception Works

Request interception shifts the browser from a passive downloader to an active gatekeeper. When you enable page.setRequestInterception(true), Puppeteer pauses every outgoing network request and triggers a request event. The browser will wait until your code explicitly tells it what to do with that specific request.

You have three primary actions available within the handler:

• request.continue(): Allows the request to proceed to the server. You can optionally modify headers or POST data here.
• request.abort(): Stops the request immediately. The browser treats this as a failed network request, preventing the asset from downloading.
• request.respond(): Bypasses the network entirely by providing a mock response (status, headers, and body) directly from your script.

Practical Implementation: Blocking Heavy Assets

To optimize for speed and cost, the most common pattern is to block resource-heavy file types. This is particularly effective for sites with heavy advertising or high-resolution imagery.

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  // Enable interception
  await page.setRequestInterception(true);

  page.on('request', (request) => {
    const resourceType = request.resourceType();
    const url = request.url();

    // Block images, stylesheets, and fonts to save bandwidth
    if (['image', 'stylesheet', 'font'].includes(resourceType)) {
      request.abort();
    } else {
      request.continue();
    }
  });

  try {
    await page.goto('https://example.com', { waitUntil: 'networkidle0' });
    console.log('Page loaded with minimal assets.');
  } catch (err) {
    console.error('Navigation failed:', err);
  } finally {
    await browser.close();
  }
})();

Execution Details

• Environment: Run this in a Node.js environment with puppeteer installed.
• Permissions: Standard user permissions are sufficient as Puppeteer controls the browser instance.
• Expected Result: The page will load significantly faster, though it will appear unstyled (no CSS) and empty of images if viewed in non-headless mode.
• Risk: Some modern web apps use CSS or specific scripts to trigger the rendering of the content you actually need. Over-blocking can result in an empty DOM.

The Critical "Hanging" Risk

The most common failure point when using interception is the forgotten request. Because Puppeteer pauses the network stack, every single request must receive a resolution. If your logic contains a conditional branch where neither continue(), abort(), nor respond() is called, the browser will hang indefinitely, and your script will eventually timeout.

Always ensure your handler ends with a catch-all request.continue() to prevent this deadlock.

Trade-offs and Performance Overhead

While blocking images reduces network latency, the act of interception itself introduces a small amount of CPU overhead. Every request must now pass through the Node.js event loop before it can be sent. For pages with hundreds of tiny requests, the overhead of the JavaScript bridge can occasionally offset the gains of blocking a few small images.

Comparison: Default vs. Intercepted

Metric	Default Load	Intercepted (No Images/CSS)
Bandwidth Usage	High (All assets)	Low (HTML/XHR only)
CPU (Browser)	High (Rendering/Parsing)	Low (Minimal Parsing)
CPU (Node.js)	Low	Moderate (Handling events)
Page Visuals	Complete	Broken/Raw HTML

Verification and Testing

To verify that your interception is working as intended, you can use the performance.now() API to measure the time between page.goto() and the resolution of the promise. Compare this against a run where setRequestInterception is set to false.

For deeper debugging, run Puppeteer with headless: false and open the Chrome DevTools Network tab. Requests that were aborted via request.abort() will appear with a status of (failed) or blocked:puppeteer, confirming the interceptor is active.