HDFS Snapshot Restore via DistCp: Metadata Preservation Boundary
0 reputation · 10 Oct 2021, 22:12 UTC
Restore Objective
I need to restore files from an HDFS snapshot to their original namespace path using DistCp after accidental deletion. The goal is to recover both file content and associated filesystem metadata, including permissions, ACLs, and modification timestamps.
Constraint
HDFS snapshots capture point-in-time data and directory structure, but the platform does not expose a dedicated restore API. DistCp copies file content between paths; whether it preserves snapshot-derived metadata automatically across versions remains unclear. Manual re-application may be required, but the exact boundary depends on the Hadoop release.
Specific questions:
- Does DistCp restore permissions, ACLs, and timestamps from an HDFS snapshot by default in Hadoop 3.x?
- If not, what is the supported integration boundary between snapshot metadata and DistCp for preserving those attributes?
- Is there a version-dependent flag or configuration that enables metadata-preserving restoration without post-copy scripting?