#multiprocessing

Fork or Spawn: Which Start Method Should a CUDA DataLoader Pipeline Standardize On?

GPU training with DataLoader(num_workers>0) carries a start-method decision PyTorch leaves to the user. Python's long-standing Linux default, fork , starts workers almost instantly and lets them inherit parent memory, including CUDA state. spawn isolates each worker in a fresh interpreter, re-importing the module and re-initializing CUDA per worker. That

Community & Careers · Editorial question

multiprocessingpytorchdataloader