Jupyter Notebook Virtual Environment Kernel Setup

Learn how to create a virtual environment, register it as a Jupyter kernel, and verify that notebooks run with the isolated dependencies you expect.

Desired Outcome

When you open a Jupyter Notebook, the notebook’s kernel should be the Python interpreter from a pre‑created virtual environment. This guarantees that import statements resolve to the exact package versions installed in that environment, keeping dependencies isolated from the system Python or other environments.

Prerequisites

Python 3.x installed and accessible via python3 (or python on Windows).
The venv module (built‑in) or virtualenv package available.
Jupyter Notebook installed either globally or inside the target environment (see caution below).
ipykernel package installed in the environment that will become the kernel.
User permissions to write to ~/.local/share/jupyter/kernels (Linux/macOS) or %APPDATA%\jupyter\kernels (Windows).

Procedure

Create the virtual environment. Choose a directory for your project and run:
```
# Unix/macOS
python3 -m venv myproject_env
# Windows (cmd)
python -m venv myproject_env
```
Replace myproject_env with your preferred name.

Activate the environment.

# Unix/macOS
source myproject_env/bin/activate
# Windows (cmd)
myproject_env\Scripts\activate
# Windows (PowerShell)
myproject_env\Scripts\Activate.ps1

Your prompt should now show the environment name, e.g., (myproject_env).

Install the kernel connector. While the environment is active, install ipykernel:
```
pip install --upgrade pip
pip install ipykernel
```
Register the environment as a Jupyter kernel.
```
python -m ipykernel install \
    --user \
    --name myproject_env \
    --display-name "Python (myproject_env)"
```
This creates a kernel spec under ~/.local/share/jupyter/kernels/myproject_env (Linux/macOS) or %APPDATA%\jupyter\kernels\myproject_env (Windows). The --user flag avoids needing administrator rights.
Deactivate the environment. You can leave the environment active or deactivate it; the kernel registration is independent.
```
deactivate
```
Launch Jupyter Notebook. Start the server from any location (no need to be inside the environment):
```
jupyter notebook
```
The server will read the kernel specs from your user directory.
Select the kernel in a notebook.
1. In the Jupyter UI, click Kernel → Change kernel.
2. Choose Python (myproject_env) from the list.
3. Verify that the kernel name appears in the top‑right of the notebook (e.g., Python (myproject_env)).

Expected Checks

Kernel presence: After step 5, run jupyter kernelspec list and confirm myproject_env appears.
Executable path: In a notebook cell, execute:
```
import sys
print(sys.executable)
```
The output should point to …/myproject_env/bin/python (Unix) or …\myproject_env\Scripts\python.exe (Windows).
Package isolation: Run !pip list in a notebook cell and compare with the output from pip list in an activated terminal of the same environment. The lists should match.
Kernel status: The kernel indicator in the notebook toolbar should show the custom name and a solid circle (indicating the kernel is alive).

Recovery Options (Rollback / Troubleshooting)

If the kernel fails to start or does not appear:

Deactivate any active environment to avoid path confusion, then re‑run the registration command from step 4.
Inspect the kernel spec: Check the generated kernel.json file:
```
# Unix/macOS
cat ~/.local/share/jupyter/kernels/myproject_env/kernel.json
# Windows (PowerShell)
Get-Content $env:APPDATA\jupyter\kernels\myproject_env\kernel.json
```
Ensure the argv array points to the correct Python binary (e.g., ["…/myproject_env/bin/python", "-m", "ipykernel_launcher", "-f", "{connection_file}"]).

Re‑register the kernel (overwrites the spec):

python -m ipykernel install --user --name myproject_env --display-name "Python (myproject_env)"

Restart the notebook server after re‑registration to pick up the updated spec.

Remove and recreate (if corruption is suspected):

# Delete the spec
jupyter kernelspec remove -f myproject_env
# Repeat steps 2‑5 to recreate

No system‑wide changes are made; removing the kernel spec only affects Jupyter’s ability to launch that environment as a kernel.

Limitations and Practical Verification

The procedure assumes you want the Jupyter server to run outside the virtual environment. Installing Jupyter inside the environment and then launching jupyter notebook from there will cause the server to use the environment’s Python, which is acceptable but may lead to duplication if you also register a kernel from the same environment. Choose one approach to avoid confusion.
Native libraries that depend on specific system packages (e.g., libblas, cuda) must be compatible with the Python version in the environment; otherwise the kernel may crash at startup with a DLL/shared‑object error. Verify by checking the environment’s python -c "import sys; print(sys.version)" matches the library’s expectations.
Changes to the environment (e.g., pip install or pip uninstall) are automatically reflected when the kernel restarts, because the kernel inherits the environment’s sys.path. No further re‑registration is needed.

To confirm persistence, stop the notebook server (Ctrl+C in the terminal), start it again, open a notebook, select the custom kernel, and repeat the executable‑path check. If the path still points to the environment’s binary, the kernel spec is correctly persisted.

Configure Jupyter Notebook to Use a Specific Python Virtual Environment Kernel