Hardening TLS: Using OpenSSL to Implement Public Key Pinning

The Trust Store Problem

Standard TLS relies on a chain of trust. Your application trusts a Root Certificate Authority (CA), which trusts an Intermediate CA, which finally trusts the server's leaf certificate. However, if a CA is compromised or a malicious actor installs a rogue root certificate on a client device, they can intercept encrypted traffic via a Man-in-the-Middle (MitM) attack. The client believes the connection is secure because the certificate is technically "valid" according to the system trust store.

Certificate pinning solves this by instructing the client to ignore the system trust store for a specific host and instead verify that the server presents a specific, pre-defined public key. The takeaway: by pinning the Subject Public Key Info (SPKI) rather than the entire certificate, you maintain security while allowing the server to renew its certificate without breaking the client connection, provided the underlying private key remains the same.

Choosing Your Pinning Level

The primary engineering decision in pinning is determining which part of the chain to lock. This is a trade-off between security granularity and operational stability.

• Leaf Pinning: You pin the public key of the end-entity certificate. This is the most secure approach but the most fragile. If the server key is rotated or compromised, every client application must be updated immediately or they will lose connectivity.
• Intermediate Pinning: You pin the public key of the CA that issues your certificates. This allows you to rotate leaf certificates frequently without updating clients, provided you stay with the same CA. It is slightly less secure because any other certificate issued by that same CA could potentially be used to spoof your server.

Extracting the Pin with OpenSSL

To implement pinning, you need a compact fingerprint of the public key. You should not pin the entire PEM file, as that is too large for efficient comparison. Instead, extract the public key and hash it using SHA-256.

Run the following commands on a workstation with OpenSSL installed (version 1.1.1 or 3.x recommended). You will need the server's certificate file (e.g., server.crt) in PEM format.

# Step 1: Extract the public key from the certificate
openssl x509 -in server.crt -pubkey -noout > pubkey.pem

# Step 2: Generate a SHA-256 hash of the DER-encoded public key
openssl rsa -pubin -in pubkey.pem -outform DER | openssl dgst -sha256 -binary | openssl enc -base64

Execution Details:

• Permissions: Read access to the certificate file is required.
• Placeholders: Replace server.crt with your actual certificate filename.
• Expected Result: A Base64 encoded string (the "pin") that your application will compare against the server's key during the TLS handshake.

Verifying the Live Chain

Before hard-coding a pin into your application, verify that the server is actually presenting the key you expect. You can use s_client to inspect the live handshake.

# Connect to the server and dump the certificate
openssl s_client -connect api.example.com:443 -showcerts < /dev/null | openssl x509 -pubkey -noout

Pipe this output into the same hashing sequence used above. If the resulting hash matches your pre-calculated pin, the server configuration is correct.

Operational Limitations and Risks

Pinning is a powerful tool, but it introduces a significant risk of "bricking" your own application. If you lose access to the private key associated with your pin, or if the key expires and you have no backup pin, your clients will be unable to connect to your server.

Mitigation Strategies:

• Backup Pins: Always generate and embed at least one "backup" pin from a separate CSR (Certificate Signing Request) stored in a secure offline vault.
• Short-lived Pins: Implement a mechanism to update pins via a secure out-of-band channel or a phased app update.
• Revocation: Be aware that pinning often bypasses standard CRL (Certificate Revocation List) or OCSP (Online Certificate Status Protocol) checks. Ensure your application still handles revocation if required by your security policy.

Verification Checklist

To ensure the pin is working, attempt to connect to the server using a proxy (like Burp Suite or Charles) that intercepts TLS traffic with its own CA. A correctly implemented pin should reject the connection immediately, as the proxy's public key will not match your hard-coded pin, even if the proxy's root CA is trusted by the OS.