Strict PTR-based Canonicalization vs. Fixed Principal Mapping for Kerberos Identity

Answer to the Questions

Long‑term operational risks of bypassing reverse‑DNS canonicalization:

• Silent identity collisions when two hosts are pinned to the same principal – audit trails and per‑host authorization become unreliable.
• Manual keytab drift: each host must keep a keytab that matches the pinned name; a host replacement or rename requires a coordinated keytab update.
• Debugging difficulty: logs show principals that no longer reflect the actual network identity, making GSSException diagnostics harder.
• Compromised revocation: if one host is compromised, all hosts sharing the principal are affected.

Impact on TLS SAN management at scale:

• With strict PTR canonicalization, SANs can be generated once per canonical FQDN – one SAN per service host – and certificate renewal is predictable.
• Fixed principal mapping forces SANs to track every pinned name change; each re‑pin or alias addition requires certificate re‑issuance, increasing operational overhead.
• When multiple hosts share a principal, a single SAN cannot represent all hosts, leading to either certificate mis‑match or the need for wildcard SANs that broaden trust.

Explanation

Strict PTR canonicalization ties the Kerberos principal to the network layer. It provides a deterministic one‑to‑one mapping: nn/host.example.com@REALM is always derived from the DNS name that the host resolves to. The downside is that every daemon depends on cluster‑wide DNS consistency; a stale PTR or mis‑configured A record can block startup with a GSSException.

Fixed principal mapping (pinning the principal or canonical hostname via krb5.conf, Hadoop configuration, or /etc/hosts) decouples the identity from the network layer. It tolerates messy DNS and multi‑homed interfaces, but the trade‑off is the loss of a unique identity per host. If two hosts use the same pinned principal, they share the same Kerberos keytab, which weakens isolation and revocation.

In practice, the stable anchor for a multi‑homed host is the canonical FQDN bound to the service’s primary interface. Pin that name consistently in both DNS (forward and reverse) and Hadoop configuration. Avoid mixing strategies across nodes – a cluster that canonicalizes on some hosts and pins on others is a common source of intermittent GSSException failures.

Verification Steps

1. On each host, confirm forward and reverse DNS agree with the principal used by the daemon:

   dig +short host.example.com
   nslookup $(dig +short host.example.com)
   

2. List keytab entries and match the host component to the resolved canonical name:

   klist -kt /etc/security/keytabs/hdfs.keytab
   

3. Enable Kerberos debug logging while starting a daemon to see the principal it requests:

   export HADOOP_OPTS="-Dsun.security.krb5.debug=true"
   $HADOOP_HOME/bin/hdfs --daemon start namenode
   

4. Verify TLS certificates match the canonical hostnames used in principals:

   openssl s_client -connect host.example.com:8443 -showcerts
   

Recommended Approach

For Hadoop 3.x clusters with multi‑homed hosts, use strict PTR canonicalization if you can maintain DNS consistency cluster‑wide. If DNS maintenance is unreliable, pin the principal but enforce a strict one‑to‑one mapping in configuration and audit that no two hosts share the same principal. Keep keytabs and TLS certificates in sync with the pinned names to avoid unnecessary certificate churn.

Missing Diagnostic Detail

To refine this recommendation, could you confirm whether your cluster currently allows multiple hosts to share a single Kerberos principal (e.g., via a shared keytab) or if each host has a unique principal? This influences the risk assessment and the choice of strategy.