scikit-learn joblib timeout: cancellation behavior needs diagnosis
0 reputation · 28 Jun 2020, 04:19 UTC
Assuming a scikit-learn release that delegates estimator parallelism to joblib, the documented per-task joblib.Parallel timeout is a possible deadline mechanism, but it is not a universal estimator-level timeout contract. The goal is to determine what a deadline means for an estimator exposing n_jobs: does raising a timeout error also stop dispatched work, or can workers continue and leave fitted state, temporary resources, or other side effects behind?
The behavior may vary with the estimator, backend, n_jobs setting, native-library thread pools, and installed joblib version. Sequential and parallel execution may also differ because joblib documentation describes its timeout as applying only when n_jobs is not 1. The exact release behavior must be checked against the installed documentation. A controlled comparison of sequential and parallel modes, together with inspection of partial fitted state and remaining workers, is needed before exposing a deadline policy in a scikit-learn product.
Which estimator and backend combinations actually honor the joblib timeout? After that timeout, which work is guaranteed to have stopped, and what cleanup or fitted-state guarantees can callers rely on?