Mastering GPU Runtime Selection in Google Colab

Google Colab users often face sudden CPU switches and session timeouts that interrupt deep‑learning workflows. This blog explains how to select, monitor, and upgrade hardware accelerators to keep notebooks running resiliently.

Choosing a GPU runtime

When you start a new notebook, open the Runtime menu, select Change runtime type, and choose GPU. Colab then attempts to assign a Tesla T4 or K80 on the free tier; Pro users gain access to P100 or V100 cards and longer maximum runtimes. The selection happens at the runtime level, not per cell, so every code cell in that session inherits the GPU driver and memory context. If demand is high, Colab may respond with a warning that no GPU is available, forcing CPU execution. A quick verification step is to insert a code cell and run !nvidia-smi. The output will list the GPU model, driver version, and total memory. If the command returns nothing or shows a CPU, the runtime did not secure a GPU. GPU (graphics processing unit, also called a hardware accelerator) is the component that accelerates matrix operations in deep learning.

Monitoring session limits

Colab does not expose a persistent GPU clock, but you can track usage from the Runtime Manage sessions page, where a GPU usage bar shows remaining time before the session is recycled. The platform automatically terminates runtimes after approximately 90 minutes of inactivity, and after reaching the maximum continuous usage limit (typically 12 hours on free tier). To mitigate loss of unsaved work, frequently save notebooks to Drive or use the built‑in checkpoint feature. A practical habit is to set a timer or alarm when approaching the 90‑minute mark, especially if you are waiting for data loading or hyperparameter tuning.

When Colab Pro makes a difference

Colab Pro subscribers pay a monthly fee for several advantages: higher‑end GPUs (Tesla P100 or V100), up to 100 hours of maximum runtime per session, and priority allocation when free‑tier GPUs are saturated. The notebook environment still auto‑installs common ML libraries TensorFlow PyTorch matched to the GPU driver version. If you need a newer CUDA toolkit or a specific library not bundled by default, you can install it with pip install inside a code cell. Keep in mind that Pro does not eliminate the inactivity timeout entirely; sessions can still disconnect after extended idle periods, so periodic !nvidia-smi checks remain useful. CUDA Compute Unified Device Architecture is NVIDIA parallel computing platform that enables GPU acceleration.

Worked example: training a small CNN

import torch
   # Verify GPU is visible
   print('GPU available:', torch.cuda.is_available())
   if torch.cuda.is_available():
       print('GPU name:', torch.cuda.get_device_name(0))
       print('Allocated memory:', torch.cuda.memory_allocated(0) / 1024**2, 'MB')
   # Simple training loop (output not guaranteed, example for illustration)
   model = torch.nn.Sequential(
       torch.nn.Conv2d(1, 32, 3),
       torch.nn.Flatten(),
       torch.nn.Linear(32*28*28, 10)
   )
   optimizer = torch.optim.Adam(model.parameters())
   data = torch.randn(64, 1, 28, 28)
   for step in range(5):
       output = model(data)
       loss = output.sum()
       optimizer.zero_grad()
       loss.backward()
       optimizer.step()
   print('Step completed on', 'GPU' if torch.cuda.is_available() else 'CPU')
   

Trade‑off and limitation

Free‑tier Colab GPUs are shared resources; availability fluctuates with demand, and the 12‑hour continuous limit plus the 90‑minute inactivity timeout mean long‑running experiments require careful session management. Colab Pro removes the most severe runtime caps and grants faster GPU classes, but at a cost. The platform does not guarantee a specific GPU model even on Pro, and unsaved state is lost whenever the runtime restarts. Always verify GPU presence with !nvidia-smi before committing to a training run, and keep critical intermediate results saved externally.

Actionable closing

1. Open Runtime Change runtime type Hardware accelerator GPU and confirm the model appears in !nvidia-smi.
2. Check Runtime Manage sessions for the GPU usage bar and note remaining time.
3. If you exceed free‑tier limits or need P100 V100 performance, evaluate a Colab Pro subscription.
4. Save notebook checkpoints to Drive every few minutes, and store model checkpoints on a persistent volume.

Following this checklist reduces the chance of unexpected CPU falls and session restarts, letting you focus on model quality rather than infrastructure surprises.