Default Hadoop UI and HDFS Permissions May Enable Accidental Public Data Exposure

Security Recommendations for Hadoop Components

To prevent accidental public exposure of data, the NameNode UI should enforce SPNEGO/Kerberos authentication for enterprise environments or simple authentication for non-production clusters. HDFS world-read access is mitigated by tightening the default umask, and encryption at rest remains an optional, manual configuration.

Enforcing NameNode UI Authentication

By default, the NameNode and ResourceManager UIs often accept unauthenticated requests. To enforce authentication, configure the following in core-site.xml and hdfs-site.xml:

• hadoop.security.authentication: Set to kerberos (recommended) or simple.
• hadoop.security.authorization: Set to true to enable permission checks.
• dfs.namenode.http.authentication.enabled: Set to true to require authentication for the web interface.

Verification: Attempt to access the NameNode UI (typically port 50070 or 9870) via a private browser window. A secure configuration should return a 401 Unauthorized or redirect to a login prompt.

Blocking HDFS World-Read Access

HDFS typically uses a default umask of 022, meaning new files are created with mode 644 (world-readable). To block this without disrupting legitimate workflows, adjust the umask to 027 (group-read, world-none) or 077 (owner-only).

1. Modify the dfs.permissions.umask property in your configuration files to 027.
2. For existing data, remove world-read permissions using the chmod command:

   hadoop fs -chmod -o-r /path/to/sensitive/data

3. Enable fine-grained access control by ensuring dfs.namenode.acls.enabled is set to true (default in Hadoop 3.x).

Verification: Create a test file and check its permissions:

hadoop fs -touchz /tmp/security_test && hadoop fs -ls /tmp/security_test

Expect a mode of -rw-r----- (640) or stricter.

Encryption at Rest Status

There is no design commitment to enable encryption at rest as a default security setting in Apache Hadoop. It remains optional and disabled by default because it requires a functioning Key Management Server (KMS) and can impact performance.

To implement this, you must manually configure a KeyProvider and create encryption zones:

hdfs crypto -createZone -keyName mykey -path /secure/data

Assumptions and Constraints

These recommendations assume a standard Apache Hadoop 3.x deployment. Distribution-specific versions (such as those from Cloudera or Hortonworks) may have different default property names or integrated security managers. Tightening umask may break legacy analytics pipelines that rely on world-read access; it is recommended to test in a staging environment first.

Missing Diagnostic Detail

Which Hadoop distribution and version are you currently running? (e.g., Apache Hadoop 3.3.x, CDH, or HDP), as this determines the specific configuration file paths and available security plugins.