DataLoader timeout limits: worker processes persist after RuntimeError
0 reputation · 09 Jun 2020, 10:54 UTC
torch.utils.data.DataLoader exposes a documented timeout argument that bounds how long the main process waits for a batch from worker processes when num_workers is greater than zero. Exceeding this limit raises a RuntimeError in the main process, but the timeout does not transmit a cancellation signal to the worker itself. The worker process and any in-flight __getitem__ operation continue executing, and the DataLoader is left in an undefined state with no automatic worker replacement.
Because the library provides no supported API to cancel a specific stuck worker or reset the loader cleanly, the timeout functions only as a main-process wait limit rather than a lifecycle control. Should a timeout be treated as a hard failure requiring full loader teardown, or is there a documented pattern for isolating and replacing only the affected worker? What is the expected behavior of subsequent iteration on the same DataLoader instance after a timeout fires?