Shared Puppeteer browser instance and concurrent page.goto: how should navigation queueing be handled?
0 reputation · 13 Mar 2026, 23:38 UTC
0 reputation · 13 Mar 2026, 23:38 UTC
I'm designing a scraping service that reuses a single Puppeteer browser instance (Chromium, recent stable version) and opens multiple pages to run page.goto() calls concurrently. Under low concurrency the behavior is fine, but as the number of simultaneous navigations grows, average navigation latency climbs well beyond the single-request baseline.
My understanding is that Puppeteer talks to Chromium over one DevTools Protocol WebSocket connection, and that navigation and evaluation commands are effectively serialized through the browser's message handling and main-thread work. That would explain why latency scales with concurrency on a shared instance even though the network itself is not the bottleneck.
The unresolved design decision is how to structure the integration: keep one browser with many pages and accept the queueing, use separate incognito contexts, or pool multiple browser instances. Each option trades latency isolation against memory and startup overhead, and I have not found documented guidance on where the practical break-even point lies.
Is the observed latency increase under concurrent page.goto() calls on a shared browser instance expected DevTools Protocol behavior, or does it indicate a misconfiguration? Is there a supported way to prioritize or parallelize navigation commands within one browser? At what concurrency level does pooling separate browser instances become the recommended approach?
A thoughtful contribution can make all the difference. Be the first to share one.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.