What would optional memoization in grunt.file.expand mean for long-running build pipelines?

Direct Answer

An opt-in memoization layer for grunt.file.expand and grunt.file.expandMapping would reduce repeated filesystem scans when the same glob patterns are evaluated multiple times within a single build run. For long-running or watch-driven pipelines, the feature would only be safe if its cache lifetime is strictly bounded by task execution boundaries and invalidated on directory membership changes (add, remove, rename)—not merely on file content modifications. It must not outlive the watch event that triggers a rebuild, otherwise it would serve stale file lists to the very tasks the watcher just re-invoked.

Invalidation Requirements

A core memoization cache would need to respect these triggers:

• Directory membership changes: any file added, removed, or renamed under a matched directory invalidates the corresponding glob result. Content-only mtime changes on existing files are insufficient because glob results depend on directory entries.
• Explicit cache.clear() API: allows plugins or task authors to force invalidation when they perform filesystem mutations outside the watcher's view.
• Per-task-run boundaries: the safest default scope confines memoization to a single task execution. Longer lifetimes (per-target, per-process) require opt-in and explicit invalidation integration.
• Watcher event integration: if the cache survives across watch cycles, it must subscribe to add/unlink/change events from the watcher and evict affected pattern entries immediately.

Interaction with grunt-contrib-watch

The watch plugin already maintains its own file set and triggers task reruns on changes. A core cache that persists across watch events creates a race: the watcher detects a change, re-runs tasks, but the memoized expand returns the pre-change file list. To avoid this, either:

• Cache lifetime ends at each task invocation (no cross-run memoization), or
• The watch plugin calls a core grunt.file.invalidate(pattern) before re-running tasks, or
• The cache internally listens to the same watcher instance and self-invalidates on relevant events.

Without one of these, the core cache re-introduces the stale-file-list problem that currently only appears at the plugin layer.

Scoping Trade-offs

Scope	I/O Savings	Staleness Risk	Safe Default?
Per-process	Maximum (all tasks share cache)	High — survives watch cycles, requires robust invalidation	No
Per-target	Moderate (shared within multi-target task)	Medium — survives target switches within a run	Opt-in only
Per-task-run	Limited (only repeated calls within one task)	Low — cleared before next task/watch cycle	Yes

expandMapping adds a second dimension: stale src caches produce stale or orphaned destination mappings, breaking incremental-build correctness in plugins that compare mtimes.

Verification Steps

1. Profile a representative build: wrap grunt.file.expand to count calls and measure time per pattern. Confirm repeated-pattern I/O is the actual bottleneck.
2. Inspect lib/grunt/file.js in your installed Grunt version to confirm no caching exists and see how expand delegates to glob.
3. Run a watch session, add/delete files matching a pattern, and observe which layer (watcher vs. expand call) picks up the change.
4. If prototyping a memoized wrapper, invalidate on watcher add/unlink events and diff returned file lists against uncached expand results.

One Missing Diagnostic Detail

What is the typical number of grunt.file.expand calls per pattern per build run in your pipeline? If it's 1–2, memoization yields negligible benefit regardless of design.