Jupyter Server restart terminates active kernels: diagnosing kernel-state continuity during migration

Direct Answers

Minimal interruption duration: 30–120 seconds for proxy cutover plus kernel restart time. Zero-downtime kernel migration is not achievable with core Jupyter Server alone.

API reliability for user prompting: /api/kernels and /api/contents reliably enumerate active kernels and verify notebook checkpoints, but they cannot automate kernel-state save. Users must manually execute checkpoint code (e.g., %store, pickle, explicit notebook save) before the maintenance window.

Externalizing kernel state without a custom spawner: Not feasible with core Jupyter's current API surface. Options require deployment changes: Jupyter Kernel Gateway / Enterprise Gateway (kernels in separate pods), ipykernel's experimental checkpointing (IPython 8+, not production-ready), or a custom kernel manager proxying to persistent kernel processes.

Confirmed Behavior

• Jupyter Server (≥ 1.x/2.x) manages kernels as child processes. A server process restart (SIGTERM, SIGKILL, or container restart) unconditionally terminates all associated kernel processes. Kernel memory state—variables, execution history, open file handles, GPU contexts—is lost and not persisted by core Jupyter.
• GET /api/kernels returns kernel metadata (id, name, last_activity, connections). GET /api/contents returns notebook file contents including autosaved checkpoints (default interval 120 s). Neither endpoint provides a checkpoint, snapshot, or migration API for in-memory kernel state.
• JupyterHub rolling restarts with KubeSpawner or DockerSpawner start replacement pods before draining old ones, reducing login downtime for new users. Existing users' kernels are still terminated when their single-user server pod stops.

Likely Explanation (Not Yet Verified in Your Environment)

If your deployment uses a reverse proxy (nginx, Traefik, Kubernetes Service) for the cutover, TCP connection draining can be sub-second. The dominant latency is kernel process startup on the new host and client-side reconnection logic (JupyterLab/Notebook 7+ auto-reconnect). Expect 30–120 seconds total interruption per kernel.

Steps for This Migration

1. Inventory: Call GET /api/kernels to list active kernels per user. Call GET /api/contents to verify each notebook has a recent checkpoint.
2. Notify: Send a maintenance notice with the kernel list and a deadline. Instruct users to run manual checkpoints (%store -r, pickle.dump, or File → Save and Checkpoint).
3. Drain: Stop accepting new kernel requests at the proxy. Wait for connections count to drop to zero on each kernel (or enforce a hard timeout).
4. Cut over: Switch proxy target to the new host. Start the new Jupyter Server process.
5. Reconnect: Users reload notebooks; JupyterLab auto-reconnect attempts to restart kernels. Kernel state is fresh; users restore from their manual checkpoints.

One Diagnostic Detail That Changes the Recommendation

Are you running JupyterHub with a spawner that supports named servers or kernel gateway integration (e.g., KubeSpawner with kernel_gateway_enabled)? If yes, kernels can survive single-user server restarts, and the migration strategy shifts to draining the gateway instead of the server. If no, the above maintenance-window approach is the only core-Jupyter path.