DataSpell Remote Kernel Architecture: SSH-Tunneled Jupyter for Teams

The Problem: Local IDE, Remote Compute, No JupyterHub

Data scientists need to run notebooks against GPUs, large datasets, or internal services that only exist in a secured VPC. Spinning up JupyterHub adds operational overhead — authentication, proxy config, user management, resource quotas. DataSpell's remote kernel feature solves this by tunneling the Jupyter protocol over SSH, using JetBrains' Gateway backend. The local IDE stays lightweight; the heavy compute runs on a remote Linux host with the same SSH identity you'd use for ssh user@host.

Smallest Viable Design

For a team of 3–10 engineers, the minimal setup is:

• One SSH bastion per environment (dev, staging, prod)
• Per-user SSH keys loaded into ssh-agent (no passwords in config)
• A shared conda or venv path on the remote host (e.g., /opt/envs/dataspell)
• No VPN, no JupyterHub, no extra auth layer

Each developer configures DataSpell → New Project → Jupyter Notebook → Configure Remote Kernel → SSH. The IDE runs a connectivity test, shows the remote Python version, and writes only host/port/key-path to .idea/workspace.xml — no secrets.

Trust and Data Boundaries

The remote backend runs under your SSH user. Kernel processes inherit that user's filesystem and network permissions. Notebook metadata — cell outputs, widget state, variable namespace — stays on the remote host. Only the rendered view and your edits travel over the tunnel.

This makes the remote host the trust boundary for data confidentiality. If your SSH key is compromised, an attacker gets everything that user can reach. Mitigate with:

• Short-lived SSH certificates (e.g., via HashiCorp Vault or AWS IAM)
• Per-developer OS accounts (not a shared dataspell user)
• Containerized kernels (Docker/Podman) for isolation if multiple devs share a host

How the Tunnel Works

DataSpell launches a headless JetBrains backend on the remote host. That backend spawns the Jupyter kernel and proxies stdin/stdout/WebSocket traffic back to the notebook UI over an SSH tunnel. The protocol is standard Jupyter (ZMQ over WebSocket); DataSpell just forwards it.

Heartbeat pings run every 30 seconds. On tunnel break, DataSpell attempts up to 3 reconnects with exponential backoff. If the remote backend survives, the kernel session persists — variables intact. If the backend dies, the kernel restarts and in-memory state is lost unless you checkpointed.

Operational Checks

Pre-flight (run from your laptop)

ssh -O check [contact removed]
# Exit code 0 = control master alive; non-zero = no tunnel

Required: SSH agent running with your key loaded (ssh-add -l shows it).

In DataSpell

Open the Remote Development tool window (View → Tool Windows → Remote Development). Expand your connection → Backend Logs. Look for:

Jupyter server started on port 4321
Tunnel established: localhost:4321 → remote:4321

Verify Kernel Identity

Run a cell:

import os, socket
print(f"PID: {os.getpid()}, Host: {socket.gethostname()}")

The PID should change only after a full kernel restart (Kernel → Restart), not after a tunnel reconnect.

Failure Modes and Mitigations

Failure	Symptom	Mitigation
Idle TCP timeout (<30 s)	Tunnel drops despite heartbeats	Set ClientAliveInterval 15 and ClientAliveCountMax 3 in /etc/ssh/sshd_config on remote; or ServerAliveInterval 15 in local ~/.ssh/config
SSH agent not forwarded	Remote git pull or private pip index fails silently	Enable ForwardAgent yes in ~/.ssh/config for the bastion; verify with ssh -A user@host 'ssh-add -l'
Backend crash (OOM, segfault)	Kernel restarts, variables gone	Checkpoint long runs: joblib.dump(state, '/tmp/checkpoint.pkl') or Jupyter %store magic
glibc too old (CentOS 7)	Backend fails to start	Use a container sidecar (Ubuntu 22.04 base) or upgrade toolchain; DataSpell 2024.2+ requires glibc ≥ 2.17
Shared OS user	Devs see each other's variables	Enforce per-developer accounts or containerized kernels (--user flag)

Simulating a Network Glitch (Diagnostic)

On the remote host (requires root):

sudo tc qdisc add dev eth0 root netem loss 50%
# ... observe reconnect toast in DataSpell ...
sudo tc qdisc del dev eth0 root

Expected: reconnect toast appears, kernel survives if backend stays up. PID from the verification cell stays same. If PID changes, backend restarted — check backend logs for OOM or segfault.

When This Design Breaks

Change the architecture if:

• Audit requirements demand centralized auth (SSO, MFA) — add JupyterHub or a reverse proxy with OIDC in front.
• Resource quotas per user — JupyterHub + KubeSpawner or batch scheduler (Slurm) needed.
• Windows remote hosts — DataSpell's remote backend supports WSL2 and native Linux only; Windows SSH server works but kernel spawn is flaky.
• Air-gapped environments — SSH bastion can't reach JetBrains license server; use offline activation or a license proxy.
• Large teams (>20) — Shared bastion becomes a bottleneck; move to per-project Kubernetes pods with dev containers.

Quick Verification Checklist

1. ssh -O check succeeds from developer machine
2. Remote Python interpreter matches project config (DataSpell shows version on connect)
3. Backend logs show "Jupyter server started" and "Tunnel established"
4. Kernel PID stable across simulated packet loss
5. .idea/workspace.xml contains only host/port/key-path — no passwords, no tokens

Limitations

• No built-in secret injection — use ssh-agent or environment variables set in remote .bashrc.
• No GPU auto-detection — ensure nvidia-smi works in the remote shell before configuring the kernel.
• Widget extensions (ipywidgets, bqplot) require the same package versions on both ends; pin in requirements.txt.
• Debugger attach works but adds latency; for step-through debugging, consider running the debugger server on the remote host and forwarding its port separately.