Iteration limit handling and ConvergenceWarning in scikit-learn iterative estimators

Answer the exact question

When fit exceeds max_iter without meeting tol, scikit‑learn emits a single ConvergenceWarning after the optimization loop finishes. The warning is the same for all solvers (LBFGS, SAG, Saga, LibLinear, SGD‑based) – it is not emitted per‑iteration and its timing does not differ across solvers. There is no built‑in way to abort fit from within the same thread by catching this warning; the call remains blocking and the estimator does not expose a cancellation flag.

Confirmed facts

• The warning is raised once per fit when the loop ends because n_iter == max_iter and the tolerance condition is not satisfied.
• Solver‑specific nuance: LBFGS (via scipy.optimize) may suppress the warning if the optimizer returns a status indicating the iteration limit; SAG/Saga and LibLinear check tolerance after each epoch and emit the warning once the epoch count reaches max_iter. SGD‑based estimators behave similarly to SAG/Saga.
• Warnings can be filtered globally (warnings.simplefilter('ignore')) so relying on them for programmatic control is unreliable.
• The estimator does not provide a callback or flag to stop fit mid‑execution; the only way to halt it is to terminate the surrounding process or thread.
• Partial results are accessible via the n_iter_ attribute (number of iterations actually performed) and, for estimators that support warm_start=True, by re‑invoking fit with a lower max_iter.

Likely explanation (based on observed behavior)

Because the warning is emitted after the solver’s internal loop finishes, its frequency is invariant across solvers – each solver performs its own iteration counting, but the scikit‑learn wrapper consolidates the result into a single warning. The variation you might see in practice (e.g., warning appearing earlier for LBFGS) stems from the underlying optimizer’s internal status codes, not from scikit‑learn emitting multiple warnings.

Recommended patterns for a wall‑clock timeout

1. Run fit in a separate process using multiprocessing.Process or concurrent.futures.ProcessPoolExecutor. Start the process, join with a timeout, and if the timeout expires call terminate() (or kill()) followed by join() to clean up.
2. Inspect partial results after termination:
   • If the process was joined before termination, the estimator’s n_iter_ reflects the iterations completed.
   • If the process was terminated abruptly, you can still recover partial coefficients by re‑fitting with warm_start=True and setting max_iter to the last known n_iter_ (you may need to store this value in a shared variable or retrieve it from a checkpoint).
3. Thread‑based timeout (only for I/O‑bound work) is not recommended because NumPy releases the GIL only intermittently; a thread may not respond to threading.Timer promptly.
4. Example code (process‑based timeout):

   import warnings
   from sklearn.linear_model import LogisticRegression
   from multiprocessing import Process, Queue
   import numpy as np
   
   def worker(X, y, max_iter, tol, q):
       with warnings.catch_warnings():
           warnings.simplefilter('always')  # ensure we see the warning
           clf = LogisticRegression(max_iter=max_iter, tol=tol, solver='lbfgs')
           clf.fit(X, y)
           q.put((clf, clf.n_iter_))
   
   X = np.random.rand(1000, 20)
   y = np.random.randint(0, 2, 1000)
   q = Queue()
   proc = Process(target=worker, args=(X, y, 1, 1e-4, q))
   proc.start()
   proc.join(timeout=2)  # wall‑clock limit
   if proc.is_alive():
       proc.terminate()
       proc.join()
       print('Fit terminated due to timeout')
       # Optionally retrieve partial results if you saved them elsewhere
   else:
       clf, n_iter = q.get()
       print(f'Finished normally, n_iter_={n_iter}')
   

Missing diagnostic detail

To decide whether the warm‑start recovery path is viable, we need to know if the specific estimator you are using supports warm_start. If it does not, partial coefficients cannot be reused after a timeout.