Diagnosing Puppeteer Navigation Timeouts: A Step-by-Step Guide

Recognizable Condition

page.goto() or page.waitForNavigation() throws a TimeoutError (default 30 seconds) even though the browser appears to have loaded content. In the DevTools Network panel you see stalled or pending requests that never complete.

Cause and Diagnostic Table

Ordered Checks

1. Switch to domcontentloaded — add { waitUntil: 'domcontentloaded' } to page.goto(). If the timeout disappears, the issue is the load event waiting for resources that never finish (common with SPA hydration).
2. Log request lifecycle — attach listeners:

   page.on('requestfailed', req => console.error('FAILED:', req.url(), req.failure().errorText));
   page.on('requestfinished', req => console.log('FINISHED:', req.url()));

   Run the navigation again and note which URLs stall or fail.
3. Disable the navigation timeout temporarily — page.setDefaultNavigationTimeout(0) — and capture page.url() after each framenavigated event:

   page.on('framenavigated', frame => {
     if (frame === page.mainFrame()) console.log('Main frame URL:', frame.url());
   });

   This reveals redirect chains and the final URL.
4. Run headed with DevTools — launch with { headless: false, devtools: true } to watch the Network and Console panels in real time.
5. Manual ready-state guard — if the above steps are inconclusive, await a custom condition:

   await page.waitForFunction(() => document.readyState === 'complete', { timeout: 15000 });

Fixes Tied to Findings

1. SPA Hydration Race

Use waitUntil: 'domcontentloaded' for the initial shell, then await a selector or a hydration marker:

await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#app-root[data-hydrated="true"]');

2. Network Idle Hang

Replace networkidle0/2 with page.waitForNetworkIdle() (Puppeteer ≥21):

await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForNetworkIdle({ idleTime: 500, timeout: 10000 });

For older versions, implement a custom Promise.race that resolves after a quiet period.

3. Cross-Origin Redirect Chain

Await navigation on the main frame after the redirect:

await page.goto(initialUrl, { waitUntil: 'domcontentloaded' });
await page.mainFrame().waitForNavigation({ waitUntil: 'domcontentloaded' });

4. Service Worker Interception

Bypass the service worker for the session:

await page.setBypassServiceWorker(true);

Alternatively, register a no-op service worker in your test setup. Clear storageState between tests if you reuse contexts.

5. Headless Auth Redirect Loop

Launch with relaxed cookie policies for local diagnosis only:

const browser = await puppeteer.launch({
  headless: true,
  args: [
    '--disable-features=SameSiteByDefaultCookies',
    '--disable-web-security'
  ]
});

Never ship these flags to CI or production.

Escalation Criteria

• If navigation succeeds in headed mode but fails headless with identical arguments, file a Chromium bug with a minimal reproduction (HTML + Puppeteer script).
• If timeouts only appear under load (many concurrent pages), profile with --enable-precise-memory-info and check for file-descriptor exhaustion (ulimit -n).

Verification Steps

1. Create a minimal test page that performs two cross-origin redirects and serves an SPA shell. Run the ordered checks and confirm each fix resolves its failure mode.
2. Execute the Puppeteer repository’s examples/navigation.js with HEADLESS=true and HEADLESS=false to compare behavior.
3. Use node --trace-warnings to surface unhandled promise rejections from navigation promises.
4. Check the Puppeteer changelog for waitForNetworkIdle API stability (added v21.0.0, stabilized v22).

Limitations and Notes

• waitUntil: 'networkidle0' is deprecated in Puppeteer v22; migrate to waitForNetworkIdle.
• page.setDefaultNavigationTimeout(0) disables guardrails — always restore the default after debugging.
• Disabling web security or SameSite cookies is strictly for local diagnosis.
• Service worker bypass affects caching; isolate test storage to avoid cross-test contamination.
• Frame navigation promises resolve on the frame that initiates navigation, not necessarily the top frame after redirects.