Diagnosing TLS Certificate Chain Verification Failures with OpenSSL s_client and verify

Recognizable condition

When a TLS handshake fails, OpenSSL‑based tools often print messages such as:

• "unable to get local issuer certificate"
• "self-signed certificate in certificate chain"
• "certificate verify failed"
• "hostname mismatch"
• "certificate has expired"
• "ee key too small" or "CA signature digest algorithm too weak"

These errors can appear while browsers or other clients still succeed, indicating a chain‑ or trust‑store problem rather than a protocol‑level block.

Cause / diagnostic table

Observed symptom	Likely cause	What to look for in s_client output
Only leaf certificate shown, verify error "unable to get local issuer certificate"	Server omits intermediate certificates	Chain depth stops at 0; no further certificates printed after the leaf.
Wrong certificate returned, hostname mismatch	Missing or incorrect SNI	The certificate presented does not match the -servername value; check the subject and Subject Alternative Name fields.
Verification fails locally but browsers succeed	Local trust store missing or stale	Using a current -CAfile bundle changes the verify return code to 0 (ok).
Error "certificate has expired"	Leaf or intermediate expired/not‑yet‑valid	Dates shown by openssl x509 -noout -dates are outside the current time.
Error "hostname mismatch" despite correct name in URI	Name absent from SAN (commonName ignored)	x509 -noout -ext subjectAltName does not list the hostname.
Error "ee key too small" or "CA signature digest algorithm too weak"	OpenSSL 3.x security level 2 rejects weak keys or SHA‑1	Key size < 2048 bits RSA or signature algorithm SHA‑1 appears in the certificate details.

Ordered checks

1. Run s_client with SNI and chain display

   openssl s_client -connect host:443 -servername host -showcerts

   Run from any host with network access to the target. No special privileges are needed; the command only opens a TCP connection and reads the TLS handshake.

   Check the printed chain: each certificate is prefixed with a depth number. Look for the final Verify return code line.

2. Interpret the verify error

   Note the error string and numeric code (e.g., num=20). If the code is 0, the chain verified; any non‑zero indicates a problem.

3. Inspect each certificate

   Save the leaf (depth 0) to a file, then examine it:

   openssl x509 -noout -subject -issuer -dates -ext subjectAltName < leaf.pem

   Repeat for any intermediates you captured.

4. Test with a known‑good CA bundle

   If you suspect a missing trust store, point OpenSSL at a current bundle:

   openssl s_client -connect host:443 -servername host -showcerts -CAfile /etc/ssl/certs/ca-certificates.crt

   On macOS you might use /etc/ssl/cert.pem; on Windows the path varies by OpenSSL build.

5. Separate chain building from trust verification

   Use the offline verify command:

   openssl verify -CAfile <trusted‑root> -untrusted <intermediates> <leaf>

   The -untrusted flag supplies certificates that the server should have sent; -partial_chain can treat an intermediate as a trust anchor for isolated testing.

6. Check for weak keys or digests

   Inspect the key size and signature algorithm:

   openssl x509 -noout -text <leaf> | grep -E 'Public-Key:|Signature Algorithm:'

   Look for RSA keys <2048 bits or SHA‑1 signatures.

Fixes tied to findings

Incomplete chain

Configure the TLS server to send the leaf certificate followed by all required intermediates. The root certificate is optional; most clients already have it in their trust store.

Missing or incorrect SNI

Ensure the client sends the correct -servername value matching the virtual host. On the server side, verify that name‑based virtual hosting is configured to select the appropriate certificate block for each SNI value.

Local trust store missing or stale

Install or update the CA bundle used by OpenSSL:

• On Debian/Ubuntu: sudo apt-get install --reinstall ca-certificates
• On RHEL/CentOS: sudo yum reinstall ca-certificates
• On macOS with Homebrew OpenSSL: brew install openssl@3 and set SSL_CERT_FILE to the bundle provided by Homebrew.
• On Windows, place the bundle in the directory pointed to by SSL_CERT_FILE or update the system root store.

Expired or not‑yet‑valid certificate

Renew the affected certificate (leaf or intermediate) and redeploy the full chain. Verify the new dates with openssl x509 -noout -dates before restarting the service.

Weak key or digest (OpenSSL 3.x security level 2)

Reissue the certificate with a key of at least 2048 bits RSA (or use ECDSA) and a signature algorithm of SHA‑256 or stronger. If you must retain legacy algorithms temporarily, activate the legacy provider:

OPENSSL_CONF=/etc/ssl/openssl.cnf openssl s_client ... -provider legacy

Note that this reduces security and should be a short‑term workaround.

Escalation criteria

If the s_client and offline verify commands succeed from a clean host but a specific application still fails, consider:

• Application‑bundled trust stores (e.g., Java cacerts, NSS, or OpenSSL statically linked with a custom bundle).
• Certificate pinning or public‑key pinning inside the application.
• Corporate TLS interception or outbound proxies that present a different chain.

Capture the s_client output from the failing host and compare the served chain and verify codes to those obtained from a known‑good network path.

Version‑sensitive behavior

OpenSSL 1.1.1 is end‑of‑life (2023‑09). OpenSSL 3.x defaults to security level 2 and loads algorithms via providers; legacy algorithms require the legacy provider. Some x509 options such as -ext subjectAltName are available only in newer releases. Always check openssl version -a to see the default CAfile, CApath, and enabled providers before applying version‑specific steps.

Limitations and practical verification

The s_client command shows exactly what the server sends during the handshake, but it does not enforce strict chain ordering; a local verify pass does not guarantee that every strict client will accept the server’s order. To confirm a fix:

1. Repeat the original failing s_client command and verify that the Verify return code is 0.
2. Run the application or client that previously failed and check that the TLS handshake completes without verification errors.
3. If possible, capture a network trace (e.g., with tcpdump) to ensure the server is now sending the intermediate certificates.

These steps provide a concrete way to check that the remediation succeeded without relying on external online scanners, which may not see internal hostnames or custom trust stores.