Choosing Between TLS 1.2 and TLS 1.3 in OpenSSL 3.x Applications

Decision and constraints

You need to configure an OpenSSL 3.x‑based server or client to use either TLS 1.2, TLS 1.3, or both. The choice affects handshake latency, cipher‑suite security, compatibility with legacy clients, and compliance requirements (e.g., FIPS 140‑3).

Comparison of supported options

Feature	TLS 1.2	TLS 1.3
Key‑exchange methods	RSA key transport, (EC)DHE, static DH	(EC)DHE only (no static RSA)
Allowed cipher suites	CBC‑mode, GCM, ChaCha20‑Poly1305 (if enabled); requires careful hardening	AEAD only: AES‑GCM, ChaCha20‑Poly1305; no CBC, RC4, DES, 3DES, SHA‑1
Handshake round trips	2‑RTT full handshake; 1‑RTT with session tickets or IDs	1‑RTT handshake; optional 0‑RTT resumption (replay risk)
Forward secrecy	Only with (EC)DHE suites; RSA key exchange lacks FS	Mandatory for all handshakes when combined with (EC)DHE
Legacy client support	Windows 7, Java 7/8, older Android, IE 11, many IoT devices	Requires clients that implement RFC 8446 (Android 10+, iOS 13+, modern browsers)
FIPS 140‑3 approval	Approved cipher suites available via FIPS provider	AEAD suites align naturally with FIPS‑approved algorithms

Trade‑offs

• Performance: TLS 1.3’s 1‑RTT handshake cuts latency by roughly 30‑50 % compared with a full TLS 1.2 handshake. 0‑RTT can further reduce latency but must be limited to idempotent operations to avoid replay attacks.
• Security: TLS 1.3 removes obsolete algorithms and mandates forward secrecy, reducing the attack surface. TLS 1.2 can be secured only by restricting to strong suites (e.g., HIGH:!aNULL:!MD5:!3DES) and disabling RSA key exchange.
• Compatibility: If you must serve clients that lack TLS 1.3 support (e.g., Windows 7, Java ≤ 8, older Android), keep TLS 1.2 enabled. Disabling TLS 1.2 entirely will break those connections.
• Compliance: Both versions are supported by the OpenSSL 3.x FIPS provider, but TLS 1.3’s AEAD ciphers map more directly to modern FIPS‑approved algorithms, simplifying validation.

Concrete implementation

Below is a minimal server‑side configuration in C that restricts the protocol version based on a decision made at runtime. Adjust the version limits according to your policy.

#include <openssl/ssl.h>
#include <openssl/err.h>

SSL_CTX* create_ctx(int allow_tls12)
{
    const SSL_METHOD *method = TLS_server_method();
    SSL_CTX *ctx = SSL_CTX_new(method);
    if (!ctx) {
        ERR_print_errors_fp(stderr);
        return NULL;
    }

    /* Load certificate and key – replace with your files */
    if (SSL_CTX_use_certificate_file(ctx, "server.crt", SSL_FILETYPE_PEM) <= 0 \
        || SSL_CTX_use_PrivateKey_file(ctx, "server.key", SSL_FILETYPE_PEM) <= 0) {
        ERR_print_errors_fp(stderr);
        SSL_CTX_free(ctx);
        return NULL;
    }

    /* Set protocol version range */
    if (allow_tls12) {
        SSL_CTX_set_min_proto_version(ctx, TLS1_2_VERSION);
        SSL_CTX_set_max_proto_version(ctx, TLS1_3_VERSION);
    } else {
        SSL_CTX_set_min_proto_version(ctx, TLS1_3_VERSION);
        SSL_CTX_set_max_proto_version(ctx, TLS1_3_VERSION);
    }

    /* Prefer strong cipher suites – OpenSSL 3.x provider‑native format */
    SSL_CTX_set_ciphersuites(ctx, "TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256");
    /* For TLS 1.2 fallback, also set a strong cipher string */
    SSL_CTX_set_cipher_list(ctx, "HIGH:!aNULL:!MD5:!3DES");

    return ctx;
}

Where to run: compile with a development toolchain that has OpenSSL 3.x headers and libraries (e.g., gcc server.c -o server -lssl -lcrypto). No special privileges are needed unless binding to a port <1024, in which case you need root or the appropriate capability.

Expected checks: after starting the server, connect with openssl s_client -connect <host>:<port> -tls1_2 and -tls1_3. The output should show a line like Protocol : TLSv1.2 or Protocol : TLSv1.3 confirming the negotiated version.

Validation and limitations

• Protocol verification: Run openssl s_client -connect example.com:443 -tls1_3 and look for Protocol : TLSv1.3. If the handshake fails, the server likely does not support TLS 1.3 or the client is restricted.
• Key material for debugging: Set SSLKEYLOGFILE=/tmp/keys.log before invoking openssl s_client; then load the log in Wireshark to confirm the selected cipher suite.
• Limitations:
  • TLS 1.3 does not allow custom curves (e.g., Brainpool); only NIST P‑256/384/521, X25519, X448 are permitted.
  • 0‑RTT data is vulnerable to replay; enable it only for idempotent requests and implement application‑level anti‑replay tokens if needed.
  • Some middleboxes that inspect the ServerHello may block TLS 1.3; monitor connection success rates and be prepared to fall back to TLS 1.2 in controlled environments.