HDFS Snapshot Restore via DistCp: Metadata Preservation Boundary

Answer

DistCp does not preserve permissions, ACLs, or timestamps by default when copying from HDFS snapshots to original paths in Hadoop 3.x. To restore metadata alongside file content, you must explicitly enable preservation flags.

Confirmed Behavior in Hadoop 3.x

DistCp treats the snapshot source as read-only and copies only file data unless configured otherwise. The -p flag controls which metadata attributes are preserved during the copy operation. Available options include:

• permissions – file and directory permissions
• acl – access control lists
• owner – file owner and group
• timestamp – modification time
• xattr – extended attributes
• blocksize – block size

Required Command Structure

For a complete metadata-preserving restore from snapshot to original path:

hdfs distcp -p permissions,acl,owner,timestamp,xattr /snapshot/.snapshot/<snapname>/<path> <original_path>

Key Constraints

• The destination directory must be empty or the copy will fail or overwrite metadata inconsistently.
• The executing user must have permission to set ownership (owner) and ACLs on the destination.
• Timestamps restored via -p timestamp may be subject to namenode policy overrides during copy.

Verification Steps

1. Before copy: capture current metadata with hdfs dfs -ls -l /original/path and hdfs dfs -getfacl /original/path.
2. After copy: verify restored metadata matches snapshot state using the same commands.
3. Confirm modification time with hdfs dfs -stat %y /path/file.

Version Dependency

These flags and behaviors are consistent across Hadoop 3.2+ and 3.3.x. Earlier versions (2.x) may not support all -p options, particularly xattr and acl preservation.