No performance gain from threading on CPU-bound Python tasks

Direct answer

The total absence of speedup in CPU-bound threading is primarily caused by GIL serialization, but voluntary preemption adds real overhead even when lock contention is minimal. For pure-Python kernels, threads cannot deliver parallel speedup on multiple cores. Multiprocessing removes the GIL bottleneck, but its process-creation and pickling costs mean it only pays off when per-task compute time clearly exceeds IPC overhead. A hybrid design is workable, yet it introduces queue backpressure and shared-state hazards that neither model has alone.

Confirmed facts vs. likely explanation

Confirmed CPython behavior: the GIL serializes bytecode execution, so pure-Python CPU-bound threads cannot run in parallel on multiple cores. The interpreter also preempts threads at a configurable interval (visible via sys.getswitchinterval()), which introduces context-switch overhead even without contention. The likely explanation for your zero speedup is that your kernels are pure Python and therefore spend their time waiting for the GIL, while the preemption overhead makes the threaded version marginally slower than serial execution rather than merely equal.

Where is the multiprocessing boundary?

There is no universal threshold, but the trade-off is predictable. Each multiprocessing task pays for process startup (or fork), memory duplication, and pickle serialization of arguments and return values. If the compute phase is shorter than the combined spawn-plus-serialization time, multiprocessing will be slower. As a rule of thumb, tasks in the low-millisecond range rarely win; tasks lasting tens of milliseconds or more with compact payloads usually amortize the overhead on a multi-core host. The exact boundary depends on payload size, the start method (fork vs. spawn), and core count, so measure it on your workload.

Hybrid threading and multiprocessing hazards

Using threads for blocking I/O and processes for compute is a legitimate pattern, but it creates coordination problems absent in either model alone. Threads share the same memory space, so they can race on Python objects; processes have separate memory spaces, so they require explicit queues or pipes. In a hybrid pipeline, an I/O thread that dispatches work to a process pool can become blocked if the pool's input queue fills, stalling all I/O. Conversely, a compute process cannot safely mutate state owned by the parent's threads. Keep the boundary strict: threads should only wait on sockets or files and enqueue work; processes should only compute and return results.

Safe verification steps

Use these scoped checks to confirm the diagnosis on your host:

python3 -c "import sys; print(sys.version); print('switch interval:', sys.getswitchinterval())"
python3 -m timeit -s "import threading" "..."
python3 -c "from multiprocessing import Pool; ..."

Profile the pickle cost separately from compute time using cProfile or manual timing around pickle.dumps.

One missing diagnostic detail would change the recommendation: do your numerical kernels execute as pure Python bytecode, or do they call into C/C++/NumPy routines that release the GIL? If the latter, threading may already provide parallelism, and the bottleneck lies elsewhere.