Question
Does SSL termination at the Load Balancer affect SNI validation for Rancher nodes?
Rahul Sage
0 reputation · 29 Jan 2025, 00:49 UTC
101K views
Loading…
0 reputation · 29 Jan 2025, 00:49 UTC
Rancher manages the trust chain between the management server and downstream clusters using a built-in CA or a provided Private CA. When provisioning nodes, the cattle-cluster-agent must validate the Rancher Server URL to establish a secure connection.
There is uncertainty regarding the behavior of various cloud provider Load Balancers when configured for SSL termination versus SSL pass-through. Specifically, if the Load Balancer terminates the TLS connection and forwards traffic to the Rancher manager via HTTP or a separate TLS session, it may alter the Server Name Indication (SNI) seen by the agent.
Which configuration is required to ensure that the downstream agent correctly validates the server certificate when a Load Balancer is intercepting the traffic? Does SSL termination introduce a trust mismatch that requires manual certificate injection on the worker nodes?
35275 reputation · 29 Jan 2025, 09:19 UTC
When a Load Balancer terminates SSL, the TLS handshake from the cattle-cluster-agent ends at the LB. The SNI (Server Name Indication) is sent by the agent to the LB, but if the LB then forwards traffic to the Rancher server (via HTTP or a new TLS session), the original SNI is lost to the backend unless the LB is explicitly configured to pass it through.
For Rancher, the cattle-cluster-agent validates the certificate presented by the endpoint against the Rancher Server URL provided during registration. If the LB terminates TLS and presents a certificate that does not match that URL, or is signed by a CA that the agent does not trust, the agent will fail to connect.
To ensure the downstream agent correctly validates the server certificate when an LB intercepts traffic, you must satisfy one of the following conditions:
SSL termination does not inherently require manual certificate injection on worker nodes if the LB presents a valid certificate from a publicly trusted CA that matches the server URL. However, if you are using a self-signed certificate on the LB or an internal CA, you must manually add that CA to the nodes' trust store to prevent the cattle-cluster-agent from rejecting the connection due to an untrusted chain.
You can verify the certificate chain using the following command from a worker node to see what certificate the agent actually sees:
curl -vI https:// 2>&1 | grep "Server certificate"
Check the "certificate:" field in the output to ensure the SAN matches your Rancher URL and the issuer is a trusted entity.
Use comments to ask for clarification. Post a solution as an answer.
35,275 reputation · 29 Jan 2025, 01:20 UTC
A critical distinction to consider is that SSL termination shifts the trust boundary from the Rancher manager's internal CA to the Load Balancer's (LB) certificate. When the LB terminates TLS, the cattle-cluster-agent validates the certificate presented by the LB, not the one generated by Rancher.
If the LB uses a certificate from a public CA, the agent will rely on the operating system's default trust store. However, if the LB uses a self-signed certificate or a private CA not known to the node, the connection will fail during the TLS handshake. To verify which certificate is being presented to the agent, you can run the following command from a worker node:
rancheropenssl s_client -connect <Rancher_URL>:443 -servername <Rancher_URL>If the output shows the LB's certificate instead of the Rancher-issued one, you must ensure the LB's CA is injected into the node's trust store or switch to SSL pass-through to maintain the end-to-end trust chain established by the Rancher CA.