Get-FileHash vs fc.exe for multi-TB restore verification: throughput vs early-exit trade-off
0 reputation · 26 Feb 2021, 05:01 UTC
0 reputation · 26 Feb 2021, 05:01 UTC
After restoring a 12 TB Windows Server Backup (wbadmin) snapshot to a new volume, the team needs an automated post-restore integrity check that fits within a four-hour maintenance window. Two documented approaches exist: Get-FileHash -Algorithm SHA256 (managed .NET streaming, single-threaded on PowerShell 5.1, parallelizable via ForEach-Object -Parallel on PowerShell 7+) and fc.exe (native binary compare, multi-threaded on recent Windows builds, exits on first mismatch but loads both files into memory).
The restore target includes 400,000 files ranging from 1 MB to 8 GB, with several hundred files exceeding 2 GB and a handful of alternate data streams (Zone.Identifier) that must be verified. Compare-Object is ruled out because it loads entire files into memory and corrupts the pipeline on binary null bytes. Robocopy /mir re-checks only size and timestamp, missing silent bit-flip corruption. No native cmdlet produces a reusable hash manifest for cross-session validation.
fc.exe’s early-exit on mismatch and multi-threaded comparison outweigh its memory pressure and lack of progress reporting when run at scale?Get-FileHash -Parallel (PowerShell 7.4) is used, what chunking strategy (per-directory vs fixed file-count batches) keeps CPU saturated without exhausting the runspace pool on an 8-core host?Get-FileHash per file, or a separate fc.exe pass on each named stream?A thoughtful contribution can make all the difference. Be the first to share one.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.