Websocket message size limits with large st.dataframe payloads

Data Serialization and Browser Rendering

Streamlit uses client-side virtualization for st.dataframe, which optimizes how rows are rendered in the browser. However, the full dataset must still be serialized and transmitted from the server to the client via websockets before virtualization occurs.

When handling large datasets, this architecture creates a bottleneck where the server memory may be sufficient, but the network payload exceeds practical websocket limits or triggers browser-side memory exhaustion. To mitigate this, developers often implement query bounding or manual pagination using st.session_state to slice the data before it reaches the rendering component.

There is a design trade-off between loading a single large bounded chunk into st.cache_data for fast client-side scrolling and implementing server-side pagination that re-queries the source on every page change.

Technical Uncertainties

• How does the interaction between st.data_editor modifications and cached query results affect the consistency of paginated views across script reruns?
• What is the optimal threshold for transitioning from client-side virtualization to manual server-side paging to avoid websocket payload failures?