Diagnosing GPU Availability and Session Limits in Kaggle Notebooks

Recognizable Condition

When working in a Kaggle Notebook you may observe one or more of the following symptoms:

• The notebook fails to start a GPU runtime and shows an error such as "GPU not available".
• GPU utilization reported by nvidia-smi drops to zero after a few minutes, causing training to stall.
• The session ends unexpectedly with a message like "Your session has been exhausted".
• Package installation takes unusually long or fails due to network timeout.
• Files saved outside the allowed directories disappear after a notebook restart.

Cause/Diagnostic Table

Symptom	Likely Cause
GPU not available at start	Weekly GPU hour quota exceeded or GPU runtime not selected
GPU utilization drops to zero	Session idle timeout or background processes consuming GPU
Session terminated with exhaustion message	Weekly GPU hour limit reached (~9 h) or maximum session length hit
Package install slow/fails	Network throttling or disk quota (<5 GB) exceeded
Output files disappear after restart	Data written outside persistent paths (/tmp or /kaggle/working)

Ordered Checks

1. Verify GPU detection

   Run in a notebook cell:

   !nvidia-smi

   Expected output includes GPU name, driver version, and memory usage. If the command returns "command not found" or shows no GPU, the runtime is not GPU‑enabled.

   Risk: None; read‑only command.

2. Check remaining GPU quota

   Open the notebook sidebar → Account → GPU Usage. Note the "Hours used this week" and the limit (≈9 h).

   If the used hours are at or near the limit, GPU access will be suspended until the next weekly reset.

3. Monitor session timer

   Look at the top‑right toolbar; a clock icon shows elapsed session time. Kaggle imposes a maximum continuous session (typically 6‑12 h).

   If the timer is close to the limit, expect automatic termination.

4. Inspect disk usage

   Run:

   !df -h

   Check the usage of /tmp and /kaggle/working. If either exceeds ~5 GB, further writes may fail.

   Risk: None; read‑only.

5. Test persistence

   Create a test file in the allowed directory:

   !touch /kaggle/working/persistence_test.txt
   !ls -l /kaggle/working/persistence_test.txt

   Save the notebook, then restart the runtime (Kernel → Restart). After restart, run the ls command again. The file should still be present.

   If the file is missing, you are writing outside the persistent paths.

Fixes Tied to Findings

• GPU not available
  • Ensure you selected GPU accelerator: Sidebar → Settings → Accelerator → choose GPU.
  • If the quota is exhausted, wait until the weekly reset (UTC Sunday) or reduce GPU‑intensive work.
• GPU utilization drops to zero
  • Keep the notebook active: avoid long idle periods; run a trivial cell (e.g., print('alive')) every few minutes.
  • Check for background processes that may preempt the GPU with !nvidia-smi and kill unnecessary ones if you have permission (rare in Kaggle).
• Session terminated with exhaustion message
  • Monitor GPU usage via the sidebar; if you are near the weekly limit, switch to CPU for lighter tasks or schedule heavy training after the reset.
  • Consider breaking training into shorter runs and checkpointing model state to /kaggle/working to resume later.
• Package install slow/fails
  • Use the Kaggle‑provided mirror where possible (e.g., pip install --index-url https://pypi.anaconda.org/kaggle/simple package).
  • Clean /tmp before large installs: !rm -rf /tmp/* (ensure no needed files).
  • If disk quota is the issue, remove unused datasets or models from /kaggle/working.
• Output files disappear after restart
  • Always write outputs to /kaggle/working or /tmp. Example: !cp model.h5 /kaggle/working/model.h5.
  • Verify persistence with the test file method described in the checks.

Escalation Criteria

If after performing the checks and applying the fixes you still encounter:

• GPU runtime fails to start despite available quota and correct accelerator selection.
• Session terminates well before the reported GPU hour limit.
• Repeated disk‑quota errors after cleaning /tmp and /kaggle/working.

Then:

1. Log out of Kaggle and log back in to reset any stray session state.
2. Visit the Kaggle Help → Contact Support page, providing:
   • Notebook URL (or ID).
   • Timestamp of the issue.
   • Output of !nvidia-smi and !df -h from the failing cell.
   • Screenshot of the GPU usage sidebar.
3. If the problem is account‑wide (e.g., GPU access blocked for all notebooks), request a quota review via the same support channel.

Limitations and Practical Verification

This guide covers the most common, documented constraints of Kaggle Notebooks as of the knowledge cutoff. It does not guarantee resolution of every edge case (e.g., temporary infrastructure incidents). To verify that a fix worked:

• After adjusting accelerator or quota usage, rerun !nvidia-smi and confirm GPU memory is allocated.
• After a session restart, check that your test file persists in /kaggle/working.
• After cleaning disk space, run a small package install (e.g., !pip install numpy) and observe completion without timeout.