Treat Your Kaggle Notebook Like a Batch Job, Not a Scratchpad

The Problem: Your Notebook Works Interactively but Fails on Commit

You tweak a cell, the output looks right, and you share the notebook. A week later a colleague runs it and hits a FileNotFoundError or a shape mismatch. The code did not change; the session did. Kaggle Notebooks preserve interactive state—variables, installed packages, and files in the working directory—that disappears when the notebook is committed. Treating the notebook as a scratchpad instead of a batch job makes results fragile.

The useful takeaway: pin your inputs, read data from the read-only /kaggle/input mount, and validate the notebook in commit mode before you trust it.

Read from /kaggle/input, Not Your Scratch Directory

Kaggle mounts every attached dataset under /kaggle/input. That mount is read-only: your notebook can read the files, but it cannot write back to them. Anything you upload during an interactive session or write to the default working directory is ephemeral. When the notebook commits, those files may not exist.

Data discovery should target the input mount explicitly. A path like /kaggle/input/my-dataset/train.csv is stable across sessions as long as the dataset version is pinned. A relative path like train.csv is not, because it depends on the current working directory and on files that may have been created interactively.

Commit Mode Is the Closest Thing to CI on Kaggle

Choosing Save and Run All (commit) executes the notebook from top to bottom in a fresh session using the dataset snapshot and accelerator you selected. Variables from earlier interactive runs, ad-hoc pip install commands, and files outside the working directory are not guaranteed to persist. This makes a commit the closest Kaggle equivalent to a continuous integration run: it proves the notebook works from a clean state, not just from the state you left behind.

Interactive iteration is still the right place to explore. The mistake is trusting interactive output as final without a commit.

Pin Inputs by Slug, Not Display Name

Dataset display names can change. Slugs—the stable identifier in the dataset URL—do not. When you attach a dataset to a notebook, Kaggle records the specific version. If the dataset author publishes a new version later, your notebook continues to read the snapshot you selected unless you manually update it.

If your environment permits internet access, the Kaggle command line interface can create dataset versions and download a dataset by slug and version. In many notebook contexts internet is disabled by default, so the attachment UI remains the primary way to pin a snapshot. Either way, the principle is the same: reference the slug and version metadata, not a friendly name that could be renamed tomorrow.

A Diagnostic Notebook You Can Commit

Before you trust a notebook, commit a small diagnostic version that prints the resolved input paths, accelerator availability, and key package versions. Then diff the interactive log against the committed log. If they differ, your interactive session was hiding state.

import os, sys, platform, importlib

def pkg_version(name):
    try:
        mod = importlib.import_module(name)
        return getattr(mod, "__version__", "unknown")
    except ImportError:
        return "not installed"

print("Python:", sys.version.split()[0])
print("Platform:", platform.platform())

input_root = "/kaggle/input"
if os.path.isdir(input_root):
    for root, dirs, files in os.walk(input_root):
        rel = os.path.relpath(root, input_root)
        print(f"[input] {rel}/ ({len(files)} files)")
        for f in sorted(files)[:5]:
            print(f"    {f}")
else:
    print(f"[input] {input_root} not found")

try:
    import torch
    print("torch:", torch.__version__)
    print("cuda available:", torch.cuda.is_available())
    if torch.cuda.is_available():
        print("device:", torch.cuda.get_device_name(0))
except ImportError:
    print("torch: not installed")

for p in ["numpy", "pandas", "sklearn"]:
    print(f"{p}: {pkg_version(p)}")

Run this interactively, then commit it. The committed log should show the same /kaggle/input tree, the same package versions, and the same accelerator status. If the committed run reports a missing package or an empty input directory, you have found the reproducibility gap before it affected real results.

The Trade-off: Reproducibility Costs Quota

Commits give you reproducibility, but they consume accelerator quota and take longer than interactive runs because they start from a cold session. Committing after every cell edit is wasteful. The practical workflow is to iterate interactively for exploration, then commit only meaningful milestones—when inputs change, when dependencies change, or before you share the notebook.

Committed output is stored with that notebook version and is useful for logs and artifacts, but it is not a general model registry or long-term artifact store. Treat it as evidence that the run happened, not as your deployment target.

Actionable Checklist

• Attach datasets and reference them by /kaggle/input/<slug>/..., never by display name alone.
• Pin the dataset version in the notebook input settings before committing.
• Commit a diagnostic notebook that prints input paths, accelerator status, and package versions.
• Diff the interactive and committed logs; resolve any mismatch before trusting the notebook.
• Reserve commits for milestones to conserve accelerator quota.

Exact GPU and TPU quotas, session timeouts, and commit limits change over time, so verify the current limits shown in the notebook interface before designing a workflow around them.