Choosing a GPU or TPU in Google Colab: How Runtime Selection and Timeouts Shape Your Workflow

Problem: Unpredictable runtimes and lost progress in Colab notebooks

When you spin up a Colab notebook, you often want a GPU or TPU to accelerate training, but the runtime you get can change, and the session can drop after a short period of inactivity. The result is a lost model checkpoint, wasted compute time, and a frustrated developer. Understanding the mechanics of runtime selection and idle‑timeout behavior lets you design a workflow that keeps your work safe and your training predictable.

Thesis: The key to reliable Colab experiments is to treat the runtime as an elastic, preemptible resource and to persist everything that matters outside of the VM’s /content directory.

1. Selecting the right accelerator

Colab offers three runtime types: CPU, GPU, and TPU. You choose the type in Runtime > Change runtime type. The actual hardware model (e.g., NVIDIA Tesla T4, P100, or TPU v2/v3) depends on the compute tier (free, Pro, or Enterprise) and current demand.

To confirm the device you’re actually using, run a shell escape from a notebook cell:

!nvidia-smi

This prints the GPU model, driver version, and memory usage. If you selected a TPU, use !tpu-accel-info or !pip install --quiet tpu-nightly==0.1.0 followed by !python -c "import torch; print(torch.cuda.get_device_name(0))" to inspect the TPU details.

Note that free‑tier GPUs are preemptible: they can be taken away mid‑run. The runtime will disconnect and you’ll lose any in‑memory state. If you need a persistent GPU, consider a paid tier or a dedicated VM.

2. Understanding idle timeouts and maximum runtime

Colab’s VMs are designed for interactive work. An idle session—no running cells and no browser activity—will disconnect after roughly 90 minutes. Even if you keep a cell running, the VM can be reclaimed after a maximum of about 12 hours on the free tier and up to 24 hours on paid tiers, depending on load.

To verify the timeout in your own environment:

1. Open a notebook and do not execute any cells.
2. Wait 90 minutes and observe the “Runtime disconnected” banner.
3. Try to resume by clicking Connect and see if the runtime re‑initializes.

Because the timeout cannot be reset by background tasks, rely on explicit cell execution or a short “heartbeat” cell that runs every 30 minutes if you need to keep the session alive.

3. Persisting data: the /content pitfall

All files written to /content live only in the VM’s RAM. When the runtime restarts—whether due to an idle timeout, preemption, or manual disconnect—those files vanish. Only Google Drive mounts or external cloud storage survive the restart.

Mount Drive:

# Mount Google Drive
from google.colab import drive
drive.mount("/content/drive")

Place your checkpoints, logs, or any artifact you want to keep in /content/drive/MyDrive/colab‑artifacts. After a runtime restart, you can reload them with:

import torch
model = torch.load("/content/drive/MyDrive/colab‑artifacts/model.pt")

Use !df -h /content to check the writable space in the VM. The free tier gives you about 68 GB, but this space is volatile.

4. Trade‑offs: free vs. paid, GPU vs. TPU

• Free GPU: T4 or P4, preemptible, 12‑hour max, 90‑minute idle. Good for experimentation but not for long‑running training.
• Pro GPU: P100 or V100, less likely to preempt, 24‑hour max, 90‑minute idle. Worth it for larger models.
• TPU: Faster for TensorFlow and PyTorch models that support XLA. No preemption on paid tiers, but still subject to idle timeout.
• All tiers: /content is volatile; Drive or cloud storage is required for persistence.

When deciding, ask:

• Do I need long training runs?
• Will I checkpoint often?
• Is my model architecture TPU‑friendly?

Actionable Checklist

1. Choose the runtime type that matches your workload and budget.
2. Verify the accelerator with !nvidia-smi or TPU info.
3. Mount Google Drive and save all checkpoints there.
4. Implement a heart‑beat cell if you need to keep the session alive.
5. Set up automatic checkpointing every N steps or minutes.
6. Monitor the runtime status via the top‑right status bar; reconnect if it disconnects.
7. For production training, consider moving to a dedicated GPU instance (e.g., GCP, AWS).

By treating the Colab runtime as a disposable, preemptible resource and by persisting everything that matters outside of /content, you can avoid losing progress and make the most of the free or paid tiers available.