Managing State and Quotas in Kaggle's Free GPU/TPU Environments

The Cost of Free Compute

The primary challenge of using Kaggle Notebooks for deep learning isn't the code—it's the environment. While the free access to NVIDIA GPUs and Cloud TPUs is a massive advantage, these resources are ephemeral. A session can time out, a quota can be exhausted mid-epoch, or the kernel can crash due to memory spikes. If you treat a Kaggle notebook like a persistent local server, you will lose hours of training progress.

The technical takeaway is simple: Design for preemption. To use Kaggle effectively, you must decouple your training state from the session lifecycle using checkpointing and efficient data streaming.

Understanding the Resource Constraints

Kaggle provides accelerators (typically T4 GPUs or TPU v3-8) governed by a weekly quota. While these quotas are generous, they are finite. Once you hit your weekly limit, your notebook reverts to CPU-only mode, which is often too slow for modern neural networks.

• Session Limits: Most sessions have a hard cutoff (often around 9–12 hours). Any data not saved to /kaggle/working or an external dataset is wiped.
• Memory Pressure: You generally have limited system RAM (roughly 13–30 GB). Loading a massive CSV or image folder directly into memory will trigger an Out-Of-Memory (OOM) kill.
• Disk Space: The working directory is limited. Large datasets should be attached as read-only Kaggle Datasets rather than uploaded to the session.

Implementing a Resumable Training Loop

To survive a session timeout or quota exhaustion, you cannot rely on a single model.fit() call. You need a strategy to save the model state and resume it in a new session.

In PyTorch, this involves saving the state_dict of both the model and the optimizer. When restarting, you load these files and set the starting epoch based on the last saved checkpoint.

# Run this in a Kaggle Notebook cell
import torch
import os

# Define path in the persistent output directory
CHECKPOINT_PATH = '/kaggle/working/model_checkpoint.pth'

def save_checkpoint(state, filename=CHECKPOINT_PATH):
    torch.save(state, filename)
    print(f"Checkpoint saved to {filename}")

# Example training loop snippet
start_epoch = 0
if os.path.exists(CHECKPOINT_PATH):
    checkpoint = torch.load(CHECKPOINT_PATH)
    model.load_state_dict(checkpoint['state_dict'])
    optimizer.load_state_dict(checkpoint['optimizer'])
    start_epoch = checkpoint['epoch']
    print(f"Resuming from epoch {start_epoch}")

for epoch in range(start_epoch, total_epochs):
    # ... training logic ...
    
    # Save every few epochs to mitigate session loss
    if epoch % 5 == 0:
        save_checkpoint({
            'epoch': epoch + 1,
            'state_dict': model.state_dict(),
            'optimizer': optimizer.state_dict(),
        })

Execution Details

• Permissions: No special permissions are needed to write to /kaggle/working.
• Risk: If the notebook is not "Saved and Run All" (committed), the /kaggle/working directory is deleted when the session ends. To persist checkpoints across different sessions, you must save the notebook version or upload the checkpoint as a new Kaggle Dataset.

GPU vs. TPU: The Engineering Trade-off

Choosing between a GPU and a TPU isn't just about raw speed; it's about framework compatibility.

Feature	GPU (T4/P100)	TPU (v3-8)
Framework Ease	High (PyTorch, TF, JAX)	Medium (TF/JAX native, PyTorch via XLA)
Setup Effort	Zero (Plug and play)	High (Requires TPUStrategy/ClusterResolver)
Best Use Case	Small to Medium Models	Massive Batch Sizes / Large Transformers

TPUs offer immense throughput but require specific data pipeline optimizations (like tf.data.TFRecord) to prevent the CPU from becoming a bottleneck. For most users, the T4 GPU is the more flexible choice due to the lack of boilerplate code required for distribution.

Limitations and Verification

The free tier is a "burst compute" tool, not a production environment. It is unsuitable for multi-day training runs or deadline-critical projects because preemption is unpredictable.

How to verify your setup:

1. Check the Resource Monitor in the right-hand sidebar of the editor to ensure the GPU is actually attached and memory is not peaking.
2. Run a 1-epoch test, save a checkpoint, restart the kernel, and verify the model loads the weights correctly.
3. Monitor your weekly quota in the Settings menu to avoid sudden mid-project outages.