Question
Does changing the notebook accelerator persist custom pip installations across kernel restarts?
Ari Moss
0 reputation · 27 Oct 2023, 21:52 UTC
108.5K views
Loading…
0 reputation · 27 Oct 2023, 21:52 UTC
Kaggle Notebooks allow users to switch between CPU, GPU, and TPU accelerators via the settings pane. This transition triggers a full kernel restart to reallocate the underlying hardware resources.
While files written to the /kaggle/working directory are preserved during a kernel restart, the state of the Python environment is volatile. Specifically, there is uncertainty regarding whether packages installed via pip during a session remain available after the environment switches accelerators, or if the container is fully reset to the base image.
What is the expected behavior for runtime-installed libraries when transitioning from a CPU to a GPU environment? Does the session maintain the installed package state, or must the installation process be repeated after the accelerator change?
35275 reputation · 27 Oct 2023, 22:19 UTC
No. When you change the accelerator (CPU ↔ GPU ↔ TPU) in a Kaggle notebook, the kernel restarts and the Python environment is reset to the base image. Any packages you installed with pip install during the session are lost and must be reinstalled after the restart.
kaggle/kaggle/working persist across kernel restarts.pip install are not persisted; they disappear when the environment is re‑provisioned.pip install in the notebook, re‑run the same command after the kernel restarts.requirements.txt file and execute pip install -r requirements.txt at the start of the notebook (or after each accelerator switch).Are you installing packages via the pip install command inside the notebook, or are you using Kaggle’s “Add packages” UI? Knowing this determines whether you need to reinstall after an accelerator switch.
Use comments to ask for clarification. Post a solution as an answer.
35,275 reputation · 28 Oct 2023, 06:41 UTC
In Kaggle’s runtime, switching from CPU to GPU or TPU triggers a full container teardown. The Python environment is rebuilt from the base image, so any site‑packages added during the session are discarded. Only the /kaggle/working folder is preserved across the restart.
To keep a library available after the switch, re‑run the pip install command at the start of each notebook or place it in a startup.sh script that Kaggle executes automatically. This guarantees the package is installed in the fresh container before any code runs.
Example:
# startup.sh
pip install --quiet my‑custom‑lib