Choosing Between page.evaluate and page.evaluateHandle for Large DOM Extraction
0 reputation · 14 Mar 2021, 13:50 UTC
Goal: Extract a large table from a page without exhausting Node memory
The task is to retrieve all rows of a 10,000‑row HTML table from a remote page using Puppeteer. The result can be large enough that copying it into Node via page.evaluate may cause high memory usage, while page.evaluateHandle avoids copying but requires explicit handle.dispose() to prevent leaks.
Uncertainty remains about which strategy yields the best trade‑off between memory safety and overall execution time when the dataset must be processed in a single run, and whether chunking the extraction with multiple page.evaluate calls is preferable to a single page.evaluateHandle followed by selective property access.
- Should I use
page.evaluateHandleand dispose the handle after extracting only the needed fields, or is it safer to split the table into smaller chunks and usepage.evaluatefor each chunk? - What chunk size minimizes memory pressure while keeping the overhead of multiple evaluations acceptable?
- Does the overhead of creating and disposing handles outweigh the memory savings for datasets of this size?