ReadMeFeed / Community knowledge

#data-streaming

Real questions. Useful conversations. Find the people who know your stack.

Question1 answer537 followingPosted 17 Aug 2026, 20:17 UTC74.8K views2,116 votes

Plotly.io.stream and Python Asyncio: Managing Long-Running Stream Cancellations

Integrating plotly.io.stream with asynchronous Python pipelines requires a mechanism to terminate active data transmissions when a client disconnects or a task is cancelled. While the timeout parameter handles inactivity, it does not provide a way to signal an immediate stop to a compute-heavy data generator that is currently pushing updates to a figure. Cur

Community & Careers · Editorial question

plotly

Community & Careers

Question1 answer1 comment3,947 followingPosted 20 Mar 2021, 02:02 UTC117.8K views1,209 votes

owner commented

PyTorch DataLoader and IterableDataset: Worker-Level Data Shuffling

When scaling data pipelines for large-scale datasets, torch.utils.data.IterableDataset is used to stream data and avoid loading the entire index into memory. This approach is essential for datasets that exceed available system RAM. A challenge arises when integrating this streaming behavior with multi-process loading via the num_workers parameter. While Data

Community & Careers · Editorial question

pytorch iterabledataset dataloader