When to Flip the GPU Switch in Kaggle Notebooks (and When Not To)

You get 30 hours of free GPU time per week on Kaggle with a verified account. That sounds generous until you burn nine of them fine-tuning a model that would have trained in twenty minutes on CPU, or lose a session's work because the 9-hour runtime limit hit mid-epoch. The GPU toggle in the notebook sidebar is a real engineering decision, not a default-on setting — and the right answer depends on your workload, not your enthusiasm.

The thesis: enable the GPU only when your compute is actually matrix-bound (deep learning training, large-batch inference), and design the session around the runtime limits before you write a line of training code.

What you actually get when you enable it

Turning on the accelerator (Settings → GPU in the notebook sidebar) typically gives you a Tesla P100 or T4 with roughly 5–16 GB of GPU memory depending on allocation, alongside tighter constraints than the CPU environment in some respects:

• 9-hour continuous session limit. When the session ends, the environment resets. Anything not written to the notebook's output directory or an attached Kaggle Dataset is gone.
• ~20 GB of working disk. Enough for most competition datasets, tight for large model checkpoints plus data.
• Restricted networking. Outbound access is limited, so pulling packages from arbitrary indexes or calling external APIs often fails. Plan to attach data as Kaggle Datasets and pre-install or vendor dependencies.
• No persistent environment. Pip installs don't survive between sessions unless you use the environment persistence option — every session starts with reinstall overhead.

Verify what you got before training. Run this in a code cell (no special permissions needed):

!nvidia-smi

You should see a Tesla-class GPU, driver version, and current memory usage. If the command errors or shows no GPU, the accelerator didn't attach — don't start a long training run assuming it did.

A concrete decision: CPU or GPU?

Consider a common competition task: training a gradient-boosted tree model (XGBoost/LightGBM) on a 2 GB tabular dataset versus fine-tuning a small transformer for text classification.

For the tree model, CPU is usually the right call. LightGBM trains fast on multiple CPU cores, you keep your GPU quota in reserve, and you avoid the 9-hour cliff for long hyperparameter searches. Forcing tree training onto GPU rarely pays off at this scale and adds data-transfer overhead.

For the transformer, GPU is the difference between a 15-minute epoch and a 3-hour one. This is what the quota is for. The decision rule I use: if the workload is dominated by large dense matrix operations (neural network forward/backward passes), GPU; if it's dominated by sequential logic, tree splits, or data wrangling, CPU.

Designing around the 9-hour limit

The session limit changes how you write the notebook. A training loop that only saves at the end is a liability. Checkpoint every epoch (or every N steps) to the output directory:

import torch, os

OUTPUT_DIR = "/kaggle/working"

for epoch in range(num_epochs):
    train_one_epoch(model, loader)
    torch.save({
        "epoch": epoch,
        "model_state": model.state_dict(),
        "optimizer_state": optimizer.state_dict(),
    }, os.path.join(OUTPUT_DIR, "checkpoint.pt"))

Run this in the notebook's code cells; /kaggle/working is the writable output path that persists as the notebook's saved output. On resume, load the checkpoint and continue from epoch + 1. The risk to watch: overwriting a single checkpoint file means a corrupted write (e.g., from an interrupted save) can cost you everything — rotate between two filenames if the model is expensive to train.

Also keep an eye on the session timer in the notebook's status bar. If you're at hour eight and an epoch takes forty minutes, save state and stop cleanly rather than gambling.

The trade-off nobody mentions: quota is shared

The weekly GPU budget applies across all your notebooks and competition submissions. A weekend of experimentation plus a few GPU-scored submissions can exhaust it, and subsequent GPU requests fall back to CPU until the quota resets. Practical implication: do data exploration, feature engineering, and baseline modeling on CPU sessions, and reserve GPU sessions for the training runs that genuinely need them. Treat the quota like a build-server budget, not an always-on resource.

Limitations and how to check your setup

Three things bite people repeatedly. First, network restrictions: test early with a harmless install attempt (e.g., !pip install some-small-package) before designing a workflow that depends on external downloads. Second, environment resets: put all installs in the first cell and keep it idempotent so a fresh session is one "Run All" away from working. Third, GPU memory: a P100-class card won't fit large batch sizes for big models — reduce batch size or use gradient accumulation rather than discovering OOM errors at hour six.

Before your next session, run !nvidia-smi, confirm your checkpointing writes to /kaggle/working, and estimate epochs × time-per-epoch against the 9-hour window. Two minutes of checking beats a lost training run.