rsync --checksum vs. Post-Restore Hash Manifests for SSH Data Recovery

Data Integrity Verification during SSH-based Restores

When recovering data from a remote server via SSH, ensuring the integrity of the restored files is critical. Two common architectural patterns exist for verifying that the destination matches the source: utilizing rsync with the --checksum flag over SSH, or streaming an archive (such as tar) and verifying the result against a pre-generated hash manifest.

Trade-offs in Verification Timing

The rsync --checksum approach performs a full-content comparison during the transfer process. This eliminates the need for a separate manifest file but significantly increases I/O overhead, as both the source and destination must read all file contents to compute checksums before and during the sync.

Conversely, streaming a tar archive through an SSH pipe is generally faster for the initial transfer. However, verification is decoupled from the transport; it requires a manifest of SHA-256 hashes generated at the time of backup to be recomputed and compared against the restored files on the local disk.

Technical Constraints

• I/O Impact: rsync --checksum may double the read load on the backup server.
• Dependency: Manifest-based verification is only possible if hashes were captured and stored securely during the backup phase.
• Detection: Default rsync quick-checks (size and mtime) may miss silent data corruption.

Given a requirement for strict data provenance and integrity, which approach is more sustainable for multi-terabyte datasets? Does the I/O penalty of rsync --checksum outweigh the operational overhead of managing external hash manifests?