Register a Project Virtual Environment as a Named Jupyter Kernel

The Problem: Notebooks Running Against the Wrong Interpreter

You open a notebook, run import pandas, and get a version from your base Python installation instead of the project's pinned dependency. The notebook executes silently against whatever kernel is selected — often a global or default environment — so import resolution, type stubs, and runtime behavior drift from the reproducible environment defined in your requirements.txt or pyproject.toml. The fix is to register each project's virtual environment as a named ipykernel so the Jupyter server can offer it as a distinct kernel choice. Once registered, selecting that kernel guarantees the notebook runs with the exact interpreter and site-packages of that environment.

Desired Outcome

After completing this guide, your JupyterLab or classic Notebook interface will list a kernel named after your project (for example, Python (my-project)). Selecting it launches a kernel process whose sys.executable points inside the project's virtual environment, so every import resolves to the versions installed there. This works across JupyterLab, Notebook, and any front-end that speaks the Jupyter protocol.

Prerequisites

• Python installed and on PATH (tested with 3.10–3.12).
• A project environment created with venv, virtualenv, or conda. The environment directory must exist and contain a working python executable.
• A Jupyter server installation reachable on your machine — this can be in a separate global environment, a user install, or a shared server. You do not need Jupyter installed inside the project environment.
• Permission to run pip install ipykernel inside the target environment. On shared systems, you may need --user or a writable --prefix path.

Procedure: Register the Kernel

1. Activate the project environment.

   # Linux / macOS (bash/zsh)
   source /path/to/project-env/bin/activate
   
   # Windows (PowerShell)
   C:\\path\\to\\project-env\\Scripts\\Activate.ps1
   
   # conda
   conda activate my-project

   Verify activation: which python (or where python on Windows) should print a path inside the environment directory.

2. Install ipykernel into the environment.

   pip install ipykernel

   This pulls the kernel machinery (including ipython) into the project's site-packages. If you use conda, prefer conda install -c conda-forge ipykernel to keep the conda solver consistent.

3. Register the kernelspec.

   python -m ipykernel install --user --name my-project --display-name "Python (my-project)"

   Where to run: inside the activated environment, from any shell.
   Permissions: --user writes to your per-account Jupyter data directory (e.g., ~/.local/share/jupyter/kernels on Linux, ~/Library/Jupyter/kernels on macOS, %APPDATA%\\jupyter\\kernels on Windows). No sudo/admin required.
   Placeholders: replace my-project with a short, unique kernel identifier (no spaces). The --display-name is the human-readable label shown in the UI; quotes are required if it contains spaces.
   Risk: if a kernel with the same --name already exists, it will be overwritten silently. Run jupyter kernelspec list first to check.

What Just Happened: The Kernelspec File

The command creates a directory named after --name inside the user kernels directory. Inside that directory sits kernel.json, for example:

{
  "argv": [
    "/home/user/project-env/bin/python",
    "-m",
    "ipykernel_launcher",
    "-f",
    "{connection_file}"
  ],
  "display_name": "Python (my-project)",
  "language": "python",
  "metadata": {
    "debugger": true
  }
}

The argv array records the absolute path to the environment's Python interpreter. If you later move or recreate the environment at a different path, the kernel will fail to start until you either re-register or edit this kernel.json to point at the new interpreter location.

Verification Checks

1. List registered kernels.

   jupyter kernelspec list

   Confirm my-project appears with a spec path under your user kernels directory.

2. Start Jupyter and select the kernel. Launch jupyter lab or jupyter notebook from any environment (it does not need to be the project environment). In the kernel selector, choose Python (my-project).

3. Run the interpreter check. In a new notebook cell:

   import sys
   print(sys.executable)
   print(sys.version)

   The printed executable path must reside inside your project environment (e.g., /home/user/project-env/bin/python). The version string should match the environment's Python version.

4. Compare package versions. In the same notebook:

   import pandas
   import numpy
   print(pandas.__version__, numpy.__version__)

   Run pip list (or conda list) in a terminal with the environment activated. The versions must match exactly. If they differ, the notebook is still using a different environment.

Recovery and Maintenance

Kernel Fails to Start or Shows ModuleNotFoundError

1. Run jupyter kernelspec list and note the spec path for the problematic kernel.
2. Inspect its kernel.json: verify the first element of argv points to an existing Python executable.

   cat ~/.local/share/jupyter/kernels/my-project/kernel.json

3. If the interpreter is missing (environment deleted or moved), remove the stale spec:

   jupyter kernelspec remove my-project

4. Recreate or relocate the environment, activate it, reinstall ipykernel, and re-run the registration command.

Switching Kernels in a Running Notebook

Changing kernels via the UI restarts the kernel process. All in-memory variables, imports, and pending outputs are lost. Persist important results (write to disk, export notebook) before switching. There is no automatic state transfer between kernels.

Shared or Multi-User Servers

On a shared JupyterHub or server, --user installs kernels only for your account. To make a kernel available to all users or to bundle it with the project, use --prefix /path/to/shared/kernels (requires write access to that prefix) and ensure the Jupyter server's c.KernelSpecManager.kernel_spec_class or --kernelspec-dirs includes that path. The kernelspec then travels with the project directory if you place it under version control.

Conda Environments: A Note on Automatic Registration

When you install ipykernel into a conda environment (conda install ipykernel), conda's nb_conda_kernels extension (if present) may auto-register a kernelspec with a display name like Python [conda env:my-project]. Before running the manual registration step, check jupyter kernelspec list. If the conda kernel already exists and points at the correct interpreter, you can use it directly — no duplicate registration needed. If you prefer a custom display name, you can still register a second spec with a different --name; just keep both in sync if the environment path changes.

Limitations and Gotchas

• Environment path stability: Kernelspecs store absolute interpreter paths. Moving, renaming, or deleting the virtual environment breaks the kernel. Keep environment paths stable or script re-registration as part of your environment setup.
• Version drift: ipykernel CLI flags and default data directories can change between releases. Run python -m ipykernel install --help to confirm flags for your version before scripting.
• Multiple Python versions: If you maintain environments for different Python versions (e.g., 3.10 and 3.11), give each kernel a distinct --name that includes the version (e.g., my-project-py310) to avoid collisions.
• No rollback for registration: The registration command writes files; there is no built-in undo besides jupyter kernelspec remove. Treat it as a state-changing operation.

Quick Diagnostic Checklist

Symptom	Likely Cause	Action
Kernel not listed in UI	Registration failed or wrong kernels directory	Run jupyter kernelspec list; check kernel.json exists
Kernel starts but imports wrong package versions	argv points to wrong interpreter	Edit kernel.json or re-register from correct environment
ModuleNotFoundError for installed packages	Environment missing packages or kernel uses base Python	Verify sys.executable in notebook; reinstall packages in that environment
Dead kernel on start	Interpreter path in kernel.json no longer exists	Remove spec, recreate environment, re-register

Final Verification

After any change, repeat the verification steps: list kernels, start a notebook on the target kernel, print sys.executable, and compare package versions against pip list from the activated environment. When all three match, the notebook is reliably bound to the project's dependency set.