Kaggle Notebook Accelerator Selection: GPU, TPU, or CPU?

Decision and Constraints

When you launch a Kaggle Notebook and select an accelerator, the choice locks for the session and determines memory, quota, library support, and data‑loading flow. The key constraints you’ll face are:

• Accelerator type – CPU (4 vCPU, 16 GB RAM), GPU (NVIDIA Tesla P100 or T4, 16 GB VRAM), or TPU (v3‑8, 8 cores with 128 GB HBM).
• Quota – GPU offers 30 hours per week (reset Saturday 00:00 UTC), TPU offers 70 hours per day (reset daily), CPU is unlimited.
• Internet window – Outbound network access works only for the first 15 minutes of a session; after that pip install, wget, or Hugging Face from_pretrained calls fail.
• Session lifetime – Maximum 12 hours per session; only the /kaggle/working directory survives as output; installed packages, downloaded weights, and environment state reset each run.
• Framework affinity – TPUs require TensorFlow or JAX (compiled with XLA). PyTorch/XLA on Kaggle is experimental and may lack full feature parity.

Accelerator Options at a Glance

Tier	Hardware	Memory	Weekly/Daily Quota	Best‑fit Frameworks	Typical Use‑Case
CPU	4 vCPU (Intel Xeon)	16 GB RAM	Unlimited	Any (PyTorch, TensorFlow, JAX, scikit‑learn)	Prototyping, small models, data preparation
GPU	Tesla P100 / T4 (16 GB VRAM)	16 GB VRAM	30 h / week	PyTorch, TensorFlow, JAX (CUDA)	Most CNNs, RNNs, medium‑size Transformers
TPU	v3‑8 (8 cores)	128 GB HBM (shared)	70 h / day	TensorFlow, JAX (XLA)	Large‑batch BERT, ResNet‑50+, massive matrix workloads

Trade‑offs

GPUs give the widest library coverage. You can run a standard PyTorch training loop with model.to('cuda') and no code changes. TPUs excel for large‑batch training and models compiled with XLA (e.g., BERT, ResNet) because the 128 GB HBM removes the 16 GB VRAM ceiling. The trade‑off: TPU workflows require tf.data or JAX data pipelines, and PyTorch/XLA on Kaggle may miss recent features.

Quota Management

A 30 h weekly GPU budget can vanish fast with daily 12 h sessions. TPU’s daily 70 h is generous but resets at midnight UTC, so a run crossing midnight loses remaining quota. Plan checkpoint intervals to avoid wasted hours.

Internet Window

All pip install, wget, or model‑download calls must finish within the first 15 minutes. The reliable pattern is to pre‑install dependencies or use Kaggle Datasets mounting – those reads do not count against the window.

Persistence

Only /kaggle/working survives a session end. Installed packages, downloaded weights, and any file outside that directory disappear. Save model checkpoints, logs, and artefacts you need later to /kaggle/working (or push them to a dataset output).

Concrete Validation Steps

1. Verify GPU attachment

   !nvidia-smi

   Expected output shows Tesla P100 or T4 with 16 GB memory. If you see No devices found, the accelerator didn’t attach – switch back to CPU and re‑select GPU.

2. Verify TPU runtime

   import os
   print('TPU_NAME:', os.environ.get('TPU_NAME'))
   try:
       import torch_xla
       print('torch_xla version:', torch_xla.__version__)
   except Exception as e:
       print('TPU not available:', e)

   Kaggle sets TPU_NAME (e.g., grpc://10.0.0.2:8470). A successful import prints the version; an error means the TPU backend isn’t ready.

3. Test internet window cut‑off

   import time, subprocess, sys
   time.sleep(1200)  # 20 min – only for demonstration; in practice just wait
   result = subprocess.run([sys.executable, '-m', 'pip', 'install', 'requests'], capture_output=True, text=True)
   print('returncode:', result.returncode)
   print('stderr:', result.stderr[:200])

   After the window closes the command returns non‑zero and stderr contains a network‑error message. This confirms you must pre‑load dependencies.

4. Checkpoint persistence test

   import torch, os
   model = torch.nn.Linear(10, 2)
   ckpt_path = '/kaggle/working/model_ckpt.pt'
   torch.save(model.state_dict(), ckpt_path)
   print('Saved:', os.path.getsize(ckpt_path), 'bytes')

   End the session (File → Save version or let the 12 h limit hit). Re‑open the notebook; the file should still exist at /kaggle/working/model_ckpt.pt. If it’s missing, you wrote outside the working directory.

5. Quota monitoring

   In the notebook UI, the right sidebar shows “GPU hours remaining” or “TPU hours remaining”. Refresh after each run to ensure you haven’t silently exceeded the limit – once exhausted the notebook falls back to CPU automatically.

Limitations & Practical Checks

• GPU memory ceiling – 16 GB VRAM caps batch size for large Transformers. Use gradient accumulation (accumulation_steps=4) or model sharding if you hit OOM.
• TPU PyTorch support – Experimental; expect missing torch.compile features and occasional XLA graph‑compilation failures.
• No custom Docker – You cannot bring a pre‑built image; all extra packages must be installed during the 15‑minute window or baked into a dataset.
• Session reset – Environment variables, cached datasets, and installed wheels disappear each run. Automate the install step in the first notebook cell.

Quick health‑check checklist (run once per new notebook):

1. Select accelerator → run nvidia-smi or TPU env check.
2. Install every required package in the first cell (within 15 min).
3. Mount any external data as a Kaggle Dataset (no internet needed later).
4. Write a dummy checkpoint to /kaggle/working and verify it survives a session restart.
5. Confirm quota numbers in the sidebar before launching a long training loop.

Following this flow lets you pick the accelerator that matches your model and framework, stay inside quota, and avoid the common “internet cut‑off” or “lost checkpoint” surprises.