Question
Kafka SSL hostname verification behavior during DNS failover with ssl.endpoint.identification.algorithm=HTTPS
Nia Lane
0 reputation · 06 Jan 2022, 19:53 UTC
75.9K views1
Loading…
0 reputation · 06 Jan 2022, 19:53 UTC
Kafka clients using the default ssl.endpoint.identification.algorithm=HTTPS verify the broker certificate hostname against the DNS name used at connection establishment. The verification occurs once during the initial handshake and is not repeated for the lifetime of that connection.
In environments with DNS-based failover or rolling broker updates, the IP address behind a broker's DNS name can change while client connections remain open. Existing connections continue using the original IP and certificate, but new connections will resolve the updated DNS record and may fail verification if the certificate does not match the new hostname.
There is no built-in mechanism to trigger re-verification or connection refresh when DNS records change. The JVM DNS cache TTL (networkaddress.cache.ttl) further complicates timing, as clients may continue resolving stale IPs even after DNS updates propagate.
35275 reputation · 07 Jan 2022, 03:12 UTC
When a Kafka client using ssl.endpoint.identification.algorithm=HTTPS experiences a DNS record change for a broker while a connection is active, the client does not re‑verify the hostname or re‑establish the connection. The SSL hostname check occurs only once, during the initial handshake, and the verified mapping is retained for the connection's lifetime. New connections initiated after the DNS update will resolve the updated address; if the broker's certificate does not list the new DNS name in its Subject Alternative Names (SANs), the handshake fails.
apachekafkanetworkaddress.cache.ttl controls DNS cache duration. A value of -1 means the client may never re‑resolve the broker address, even after DNS propagation.-Dnetworkaddress.cache.ttl=60 and observe whether connections re‑resolve after a manual DNS change.openssl s_client -connect broker:9092 to verify the certificate contains SANs for all potential failover DNS targets.*.broker.example.com) or synchronizing certificate deployment across all brokers.One diagnostic detail that would change the recommendation: the current networkaddress.cache.ttl value on the client runtime. Knowing this determines whether the client will attempt new DNS resolutions after a failover event.
Use comments to ask for clarification. Post a solution as an answer.
35,275 reputation · 07 Jan 2022, 00:13 UTC
Setting networkaddress.cache.ttl only controls when the JVM opens a new socket; it does not force the Kafka client to drop an established broker connection. The client keeps using the cached node from broker metadata until metadata.max.age.ms expires, the connection breaks, or a send fails. Hostname verification also applies to the host string actually dialed, which is often the value in advertised.listeners, not the public DNS name. If failover returns an IP or internal hostname missing from the broker certificate SANs, new connections fail even when public DNS resolves correctly. Check the effective metadata.max.age.ms and inspect advertised listener host strings during a metadata refresh to confirm the mismatch.