Gulp 4: gulp.lastRun() incremental builds versus full re-runs for large projects
0 reputation · 10 Feb 2023, 00:30 UTC
0 reputation · 10 Feb 2023, 00:30 UTC
Goal: Choose between incremental and full re-runs for a Gulp 4 build that processes a large TypeScript source tree.
Constraint: Faster rebuilds are needed, but outputs must not go stale when non-globbed inputs such as tsconfig.json or imported templates change.
Uncertainty: gulp.lastRun(task) with gulp.src({ since }) limits processing to files changed since the last successful task run, while omitting since processes every matched file every run. The trade-off is I/O savings against missed dependencies outside the glob and stale dist files after deletions.
Does gulp.lastRun() reliably cover config files outside the gulp.src glob when they affect task output?
Should the build use incremental since filtering for every task or reserve full re-runs for config-dependent tasks?
32775 reputation · 10 Feb 2023, 09:26 UTC
gulp.lastRun() does not automatically cover config files outside the gulp.src glob. When you use gulp.src({ since: lastRun(task) }), only files matching the glob pattern and modified since the last successful run are processed. Config files like tsconfig.json or imported templates are invisible to this filter unless explicitly added to the source array or watched separately.
gulp.lastRun(task) returns the timestamp of the last successful task run.gulp.src({ since: timestamp })` limits the stream to files modified after that timestamp.since filter, even if they affect compilation output.since; stale outputs persist without a clean step.Combine incremental filtering for source changes with a separate full rebuild triggered by config changes:
const { src, dest, lastRun } = require('gulp'); const ts = require('gulp-typescript'); function compileTs() { const tsProject = ts.createProject('tsconfig.json'); return src(['src/**/*.ts'], { since: lastRun(compileTs) }) .pipe(tsProject()) .pipe(dest('dist')); }
- Watch source files for incremental compilation:
function watchSources() { watch('src/**/*.ts', compileTs); }
- Create a full rebuild task that cleans and compiles without
since:const del = require('del'); function clean() { return del(['dist']); } function fullTsBuild() { const tsProject = ts.createProject('tsconfig.json'); return src('src/**/*.ts') .pipe(tsProject()) .pipe(dest('dist')); } const build = series(clean, fullTsBuild);
- Watch config files and trigger full rebuild on changes:
function watchConfig() { watch(['tsconfig.json', 'src/**/*.html', 'src/**/*.template'], build); }
- Export combined watchers:
exports.default = parallel(watchSources, watchConfig);Missing diagnostic detail
The recommendation assumes no pre-clean step before each incremental compile. If your current workflow already deletes
distbefore runningcompileTs, the incrementalsincefilter provides minimal benefit—consider using a simple full rebuild on every change instead for consistency.
Use comments to ask for clarification. Post a solution as an answer.
5,040 reputation · 10 Feb 2023, 03:28 UTC
gulp.lastRun() only compares file modification times for the files that actually match the glob you pass to gulp.src. It does not keep any record of the contents of files that lie outside that glob, such as tsconfig.json or template files. If one of those files changes, the incremental filter will happily skip the task, leaving stale output. A common fix is to split the build: keep source‑only tasks incremental, and wire a gulp.watch('tsconfig.json') or a dedicated “config‑watch” task to trigger a full rebuild when a config file changes.