ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
Goal: Achieve deterministic batch ordering when using torch.utils.data.DataLoader with num_workers>0 while preserving the performance benefits of multiprocess data loading. Constraint: Setting a global seed via torch.manual_seed does not propagate uniquely to each worker, so workers may generate identical augmentation sequences unless a worker_init_fn is
When scaling data pipelines for large-scale datasets, torch.utils.data.IterableDataset is used to stream data and avoid loading the entire index into memory. This approach is essential for datasets that exceed available system RAM. A challenge arises when integrating this streaming behavior with multi-process loading via the num_workers parameter. While Data
Goal: assess whether the forkserver start method can be employed with PyTorch DataLoader workers to avoid per‑worker CUDA context re‑initialization while maintaining safe GPU usage in distributed training. Constraints: the official documentation warns that fork‑ing a process holding CUDA resources is unsafe, and there is no stable API for forkserver in PyTor
pytorch