Colab runtime type switch without losing in-memory training state

Answer (Key Points)

• Colab has no API to carry in‑memory training state across a runtime type change. The VM is recreated, and all variables are lost.

• The only reliable method is to checkpoint the model weights and the optimizer state to persistent storage (e.g., Google Drive) before changing the runtime.

• To resume training exactly, checkpoint both weights and optimizer state; checkpoints of weights alone will miss learning‑rate schedules, momentum buffers, and other optimizer internals.

• Switching between GPU types (T4 → L4, etc.) triggers the same VM restart and does not preserve state; the behavior is identical to CPU → GPU transitions.

Why This Is the Case

When a user selects Runtime → Change runtime type, Colab terminates the current VM and spins up a fresh one. The new VM starts with a clean environment; no hidden persistence mechanism is exposed to notebooks. This is confirmed by assigning a Python variable, restarting the runtime, and observing that the variable is undefined afterward.

Recommended Checkpointing Workflow

1. Mount Google Drive (or another persistent bucket):

   from google.colab import drive
   drive.mount('/content/drive')

2. Save model and optimizer:

   # PyTorch example
   import torch
   checkpoint = {
       'epoch': epoch,
       'model_state_dict': model.state_dict(),
       'optimizer_state_dict': optimizer.state_dict(),
       'loss': loss_value,
   }
   torch.save(checkpoint, '/content/drive/MyDrive/checkpoint.pt')

3. Change the runtime type via the UI.

4. After the new VM starts, reload the checkpoint:

   checkpoint = torch.load('/content/drive/MyDrive/checkpoint.pt')
   model.load_state_dict(checkpoint['model_state_dict'])
   optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
   start_epoch = checkpoint['epoch'] + 1
   

5. Resume the training loop from start_epoch.

Considerations for Large Models

• Checkpointing every epoch can be I/O‑heavy; choose a frequency that balances recovery time with Drive quota usage.
• If the target GPU has less memory than the source GPU, the loaded model may fail to fit. In that case, consider checkpointing after every few epochs or using a smaller batch size.
• Always record the epoch counter and any learning‑rate scheduler state in the checkpoint to ensure exact continuity.

Same‑Accelerator Transitions

Switching from one GPU type to another (e.g., T4 → L4) behaves exactly like a CPU → GPU transition: the VM is recreated, and all in‑memory state is lost. The same checkpointing strategy applies.

Missing Diagnostic Detail

To fine‑tune the checkpoint frequency, it would help to know the size of your model and the memory capacity of the target GPU. If the model comfortably fits on the new GPU, less frequent checkpoints may be acceptable; otherwise, more frequent checkpoints or a smaller batch size may be required.