Puppeteer Launch Options and Page Request Interception: Ensuring Request Capture Across Navigations
0 reputation · 15 Nov 2021, 23:17 UTC
0 reputation · 15 Nov 2021, 23:17 UTC
Goal: guarantee that every network request is intercepted after a page navigation, including when working with incognito contexts.
Constraints: request interception must be enabled before any navigation; enabling it after a page has started loading can miss early requests. Incognito pages do not inherit the interception flag from their parent context, requiring a separate setup. Furthermore, the API does not automatically re‑enable interception after a navigation completes, which creates a race condition if a script toggles the flag mid‑navigation.
Questions: What is the recommended pattern to enable interception reliably before each navigation in both regular and incognito pages? How can developers ensure that interception state persists across navigations without manual re‑enabling? Is there a documented way to propagate the interception setting to newly created incognito contexts?
34275 reputation · 16 Nov 2021, 07:14 UTC
page.setRequestInterception(true) immediately after newPage() and before any navigation. Use this helper for both regular and incognito pages.setRequestInterception(false) resets the flag and creates a race window where early requests slip through.async function createInterceptedPage(browserOrContext, handler) {
const page = await browserOrContext.newPage();
await page.setRequestInterception(true);
page.on('request', handler);
return page;
}
// Regular page
const page = await createInterceptedPage(browser, req => req.continue());
// Incognito page
const incognitoCtx = await browser.createIncognitoBrowserContext();
const incognitoPage = await createInterceptedPage(incognitoCtx, req => req.continue());Puppeteer's setRequestInterception is a per-page CDP flag. The Chrome DevTools Protocol resets it to false on every navigation (including same-document navigations). The only documented guarantee to capture the very first request is enabling interception before any navigation command (page.goto, page.click on links, page.reload, etc.).
Incognito contexts are fully isolated: they share no cookies, storage, service workers, or interception state with the parent browser context. The helper above ensures each page—regular or incognito—starts with interception active.
puppeteer.launch({...}) cannot pre-enable interception.request.continue()) instead of setRequestInterception(false).page.waitForNavigation() or page.waitForNetworkIdle(), interception must already be active before the action that initiates navigation.{ interception: true, cacheEnabled: true } (experimental) if cache behavior matters.https://example.com, and logs every intercepted request URL.npm view puppeteer versions --json) to verify no regression in setRequestInterception persistence.Which Puppeteer version and Chrome/Chromium revision are you deploying against? Behavior can differ across CDP versions, and Firefox (via puppeteer-firefox) has different interception semantics.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.