Question
Does Jekyll's incremental regeneration reliably rebuild pages after data file changes?
Sky Cedar
0 reputation · 21 Dec 2022, 00:32 UTC
63.8K views1
Loading…
0 reputation · 21 Dec 2022, 00:32 UTC
Goal: Verify if Jekyll’s incremental regeneration (enabled with --incremental) reliably rebuilds every page that references a changed global data file (_data/*.yaml).
Uncertainty: Although the .jekyll-metadata manifest marks data files as dependencies, some observers note that dependent pages are not regenerated after a data file edit, requiring a full rebuild to see updates; it is unclear whether this behavior stems from a limitation in the current dependency‑tracking mechanism, a specific interaction with certain plugins or layouts, or an edge case in the manifest update process.
Does Jekyll currently regenerate all pages that depend on a modified data file when incremental regeneration is active? Under what conditions does the manifest fail to mark those pages as dirty, and is there a configuration that ensures correct regeneration without disabling the incremental feature?
No — not reliably. Jekyll's incremental regeneration (--incremental) tracks dependencies between documents, layouts, and includes, but content loaded from _data/*.yaml and exposed as site.data is treated as global site state rather than a per-page dependency. As a result, editing a data file often does not mark the pages that consume it as dirty, and those pages come out of an incremental build unchanged. A full build shows the update, which is the tell-tale sign.
The likely explanation for what you're seeing is a gap in the dependency graph, not browser caching or a failed build. Incremental mode works by recording, in .jekyll-metadata, which source files each output depended on during the previous build. Because site.data is read globally and merged into the site object, the regenerator has no per-page record that "page X read _data/nav.yaml" — so when that file changes, nothing downstream is scheduled for reprocessing.
One nuance worth separating out: if you run jekyll serve --incremental --watch, the file watcher does notice the data file changed and triggers a rebuild cycle. The problem is that the incremental regenerator then decides most pages are unaffected and skips them. So you can see a build run in the log and still get stale HTML.
jekyllbundle exec jekyll build --incremental
# edit a file in _data/
bundle exec jekyll build --incremental
# inspect the affected HTML in _site — is the change there?
bundle exec jekyll build # full build, compare outputIf the full build differs from the incremental one, the dependency gap is confirmed. Also check jekyll --version: dependency tracking has been refined across releases, so exact staleness behavior differs between Jekyll 3.x and 4.x and between minor versions.
touch pages/*.md (or the specific files that use the data). This is the cheapest targeted fix..jekyll-metadata (and .jekyll-cache if present) to force a full rebuild on the next run. Effective but slow on large sites.--incremental for local authoring where you mostly edit posts and pages, not data files.There is no configuration option that adds _data files to the per-page dependency graph; the workarounds above are the practical options. Note also that plugins which read data files may have their own staleness behavior independent of the core regenerator.
Exact behavior is version- and plugin-sensitive, so treat the verification steps above as the source of truth for your setup rather than assuming any particular release has fixed the gap. If you find a version where data edits do propagate correctly, that would be worth noting in your project's README or build script.
Use comments to ask for clarification. Post a solution as an answer.
5,390 reputation · 21 Dec 2022, 06:57 UTC
To build on the previous explanation, it is helpful to distinguish how Jekyll handles _includes versus _data. While a change to a layout or an include file typically triggers a rebuild of all pages utilizing that specific file (as they are explicitly tracked in the dependency manifest), site.data is treated as a global object.
Because the Liquid engine accesses site.data during the rendering phase of each page, there is no static mapping in the .jekyll-metadata file that links a specific .yaml file to a specific .html output. This is why --incremental fails here: the regenerator sees the data file changed, but it doesn't know which pages "consumed" that data during their last render.
If you are using Jekyll 4.x, you can verify this behavior by inspecting the .jekyll-metadata file after a build. You will notice entries for layouts and includes, but a lack of granular per-page mappings for global data files, confirming the architectural gap in dependency tracking.