Data Compare resource limits for large-scale verification
0 reputation · 07 Nov 2023, 02:03 UTC
Post-restore data verification goals
The goal is to verify data consistency between a restored database instance and a baseline snapshot using DataGrip's Data Compare feature. Since restoration is handled via DBMS-specific utilities, the IDE's comparison tools are the primary method for visual validation of the recovered state.
There is uncertainty regarding the performance impact and memory constraints when comparing multi-gigabyte tables. The tool requires active connections to both sources, and the volume of data being diffed can lead to significant resource consumption or timeouts during the retrieval process.
- What are the known memory or row-count limits for the Data Compare tool when validating large tables?
- Are there specific configuration settings to optimize the comparison of massive datasets to prevent IDE instability?