Pin Kaggle Dataset Versions for Reproducible ML

Why Dataset Pinning Matters

When you run a Kaggle Notebook, the platform attaches a dataset by its slug—a user‑scoped identifier like username/dataset-name. Behind the scenes, each dataset is versioned, and the Notebook stores the exact version ID it used. If you later update the dataset without changing the Notebook, the new data silently replaces the old one, causing hidden data drift. The result? Training metrics shift, model comparisons become meaningless, and debugging turns into a guessing game.

How the Pinning Mechanism Works

In the Notebook UI, when you attach a dataset you can choose a specific version from the dropdown. That choice is written into the Notebook’s metadata and preserved in every commit. The Notebook’s runtime will always load the files from that snapshot, regardless of any newer releases.

Programmatically you can achieve the same effect with the Kaggle API:

# Download a specific version of a dataset
kaggle datasets download -d username/dataset-slug -v 1.2.0

Here 1.2.0 is the immutable version ID. If you omit -v, the API fetches the latest, which is why pinning is essential.

Concrete Example: The Titanic Dataset

Suppose you publish a Notebook that trains a logistic regression model on the Titanic training set. You attach the public dataset titanic and pin it to version 2024-02-01. The Notebook’s dataset.json metadata now contains:

{"dataset_slug": "titanic", "dataset_version": "2024-02-01"}

To verify you’re using the intended snapshot, add a quick hash check:

import hashlib, pathlib
path = pathlib.Path("/kaggle/input/titanic/train.csv")
hash_val = hashlib.sha256(path.read_bytes()).hexdigest()
print("train.csv SHA256:", hash_val)

Commit this change. Later, if the dataset owner uploads a new version, the Notebook will keep running the old hash unless you deliberately update the pinned version. You can compare two runs side‑by‑side:

# Version 2024-02-01
kaggle kernels push --commit-message "Baseline model" --dataset-version 2024-02-01

# Version 2024-05-01
kaggle kernels push --commit-message "Baseline model – new data" --dataset-version 2024-05-01

Running both kernels and comparing the printed hash values and evaluation metrics will expose the data drift introduced by the newer dataset.

Trade‑offs and Limitations

• Storage overhead: Each pinned version consumes space on Kaggle’s infrastructure. If you pin many large datasets, you may hit storage quotas.
• Ownership changes: If the dataset owner renames or transfers ownership, the slug may break, causing your Notebook to fail for new viewers.
• Private datasets: A Notebook that depends on a private dataset will run only for users with explicit access. Public viewers will see a “dataset not found” error.
• Library updates: Even with a pinned dataset, changes in the runtime environment (e.g., NumPy 2.0) can alter numerical results. Full reproducibility may require containerization.

Actionable Checklist for Reproducible Kaggle Experiments

1. Always attach datasets via the UI and select a specific version.
2. Include a hash check of key input files in your Notebook.
3. Use the Kaggle API to list available dataset versions and compare against the pinned version before publishing.
4. Commit the Notebook after each dataset update and document the version change in the commit message.
5. When sharing results, provide the exact dataset version ID and the hash values used.

By following these steps, you convert the implicit “latest” pointer into a hard contract, eliminating silent data drift and making your ML experiments truly reproducible.