Memory leak in gulp.watch leaving vinyl‑fs fs.ReadStream instances unclosed

Question

The goal is to confirm whether the memory‑leak fix introduced in gulp 4.0.2 fully resolves resident‑memory growth when gulp.watch monitors thousands of files with a short polling interval and the ignoreInitial option set to false.

Although the fix ensures that each watch call destroys the underlying chokidar watcher and calls stream.destroy() on vinyl‑fs streams, reports indicate that aggressive polling or large directory trees can still cause elevated memory usage if the watcher holds onto numerous change events, and third‑party plugins that manage their own streams may bypass the cleanup.

Does enabling a polling interval of 100 ms with ignoreInitial:false lead to a measurable increase in heap size over a 10‑minute run? Does the retention of change events prevent garbage collection of vinyl‑fs streams even after stream.destroy() is called? Are there specific plugin patterns that are known to re‑introduce similar leaks despite the core fix?

Wasi · Answer

AnswerEnabling a polling interval of 100 ms with ignoreInitial:false can lead to a measurable increase in heap size over a 10‑minute run when gulp.watch monitors many files, but the growth is mainly caused by retained change events or third‑party plugins that keep their own stream references, not by a failure of the core stream.destroy() fix.Confirmed factsgulp 4.0.2’s fix ensures each watcher calls stream.destroy() on the underlying vinyl‑fs fs.ReadStream objects and destroys the chokidar watcher.When stream.destroy() is called and no other references exist, the stream becomes eligible for garbage collection.Likely explanationIf change events are stored in arrays, closures, or plugin state, the associated fs.ReadStream objects stay reachable, preventing GC even after destroy().Aggressive polling (100 ms) generates many events quickly; with large directory trees the event buffer can grow, increasing the chance that user code or plugins retain references.Some plugins create their own vinyl‑fs streams (e.g., by calling gulp.src() and storing the returned stream) or wrap streams without properly ending them, re‑introducing the leak despite the core fix.Steps to verify for your caseCreate a test folder with a known number of files (e.g., 5 000, 20 000) and a realistic depth.Write a minimal gulp task: gulp.watch('**/*', { polling: true, interval: 100, ignoreInitial: false }, (cb) => { console.log('change'); cb(); });Run Node with --inspect or use clinic.js/heapdump to capture heap snapshots before starting the watch, after 5 minutes, and after 10 minutes.Compare retained sizes of vinyl‑fs ReadStream objects; a steady increase indicates a leak.Optionally add a suspect plugin that stores the stream from gulp.src() in a global array and observe whether heap growth accelerates.Missing diagnostic detailTo refine the recommendation, please confirm the approximate number of files (and directory depth) you are watching with gulp.watch.

Memory leak in gulp.watch leaving vinyl‑fs fs.ReadStream instances unclosed

1 answer

Answer

Confirmed facts

Likely explanation

Steps to verify for your case

Missing diagnostic detail

Your perspective belongs here.

0 question comments