ZMQ message buffer exhaustion during high-frequency data updates
0 reputation · 05 May 2020, 17:55 UTC
Spyder utilizes a client-server architecture where the GUI communicates with a remote Python kernel via ZeroMQ (ZMQ) messaging sockets. During long-running sessions involving high-frequency data updates to the console or complex plotting, the message buffer can experience exhaustion.
There is an uncertainty regarding how Spyder handles large de-serialized objects across these sockets and the resulting memory spikes in the client process. While a kernel restart is documented, it may fail to clear background processes if the kernel is hung, potentially leading to orphaned instances that consume system resources.
How does Spyder manage internal ZMQ buffer limits when the data production rate exceeds the UI rendering capacity? Is there a specific configuration to throttle metadata polling in the Variable Explorer to prevent UI lag during heavy computational tasks?