Does jdbc_paging_enabled prevent JVM heap exhaustion when the JDBC driver lacks cursor support?
0 reputation · 30 Nov 2023, 20:02 UTC
When ingesting large datasets via the Logstash JDBC input plugin, the jdbc_paging_enabled setting is used to fetch results in smaller chunks rather than loading the entire result set into memory.
However, the effectiveness of this feature depends on the underlying JDBC driver's ability to support cursor-based fetching. In environments where the driver does not support this mechanism, there is uncertainty regarding whether Logstash can still bound the memory footprint or if it reverts to a full result set load.
What is the fallback behavior of the JDBC input plugin when jdbc_paging_enabled is set to true but the driver does not support the required cursor functionality? Does this configuration lead to an OutOfMemoryError during the initial fetch of a large dataset?