Choosing and Configuring TLS 1.3 Cipher Suites in OpenSSL 1.1.1+ and 3.x

The Decision

You need to decide which TLS 1.3 cipher suites to enable and how to configure them in OpenSSL. The choice affects security, performance, and compliance. Constraints include your OpenSSL version (1.1.1 vs 3.x), target hardware (AES‑NI availability), and whether you must meet FIPS 140‑3 requirements.

Supported TLS 1.3 Cipher Suites

OpenSSL 1.1.1 and later support exactly five TLS 1.3 cipher suites. All are AEAD; no CBC or RC4 suites exist in TLS 1.3.

Suite Name	Encryption	Hash	FIPS 140‑3 Approved	Typical Use Case
TLS_AES_256_GCM_SHA384	AES‑256‑GCM	SHA‑384	Yes	High‑security default; best with AES‑NI
TLS_CHACHA20_POLY1305_SHA256	ChaCha20‑Poly1305	SHA‑256	No	Mobile, embedded, or older x86 without AES‑NI
TLS_AES_128_GCM_SHA256	AES‑128‑GCM	SHA‑256	Yes	Balanced security/performance; FIPS compliant
TLS_AES_128_CCM_SHA256	AES‑128‑CCM	SHA‑256	No	Constrained IoT (RFC 6655); no hardware acceleration
TLS_AES_128_CCM_8_SHA256	AES‑128‑CCM (8‑byte tag)	SHA‑256	No	Extremely constrained IoT; lower integrity margin

Trade‑offs

AES‑GCM vs ChaCha20‑Poly1305

On CPUs with AES‑NI (most modern x86_64), AES‑GCM is faster. Without AES‑NI (ARM Cortex‑A, older x86, many embedded cores), ChaCha20‑Poly1305 outperforms AES‑GCM by a wide margin. Choose based on your deployment hardware profile.

CCM Suites

CCM modes are designed for very constrained devices (RFC 6655). They have lower throughput and no common hardware acceleration. Avoid unless a specific profile mandates them.

FIPS 140‑3 Compliance

The OpenSSL 3.0 FIPS provider only approves TLS_AES_128_GCM_SHA256 and TLS_AES_256_GCM_SHA384. ChaCha20‑Poly1305 and both CCM suites are not FIPS‑approved. If you require FIPS, you must load the FIPS provider and restrict the suite list accordingly.

Key Exchange Groups

TLS 1.3 mandates ephemeral (EC)DHE. Supported groups (in typical preference order):

• X25519 – preferred for performance and side‑channel resistance
• P-256, P-384, P-521 – NIST curves
• ffdhe2048, ffdhe3072, ffdhe4096 – finite‑field DH

Configure groups via SSL_CTX_set1_groups() (API) or the Groups directive in openssl.cnf (OpenSSL 3.0+). Place X25519 first unless compliance dictates otherwise.

Configuration Methods

OpenSSL 3.0+: openssl.cnf Directives

OpenSSL 3.0 introduces Ciphersuites and Groups directives in the [system_default_sect] section. This is the simplest way to enforce policy for all applications that use the default config.

[system_default_sect]
Ciphersuites = TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256
Groups = X25519:P-256:P-384
MinProtocol = TLSv1.3
MaxProtocol = TLSv1.3

This snippet enables TLS 1.3 only, sets the preferred suite order, and restricts key‑exchange groups. Place it in your system openssl.cnf (often /etc/ssl/openssl.cnf) or a custom config file loaded via OPENSSL_CONF.

API Configuration (All Versions 1.1.1+)

Use SSL_CTX_set_ciphersuites() for TLS 1.3 suites and SSL_CTX_set1_groups() for key‑exchange groups. The legacy SSL_CTX_set_cipher_list() does not affect TLS 1.3.

/* Example in C */
SSL_CTX *ctx = SSL_CTX_new(TLS_method());
const char *suites = "TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256";
if (!SSL_CTX_set_ciphersuites(ctx, suites)) {
    /* handle error */
}
int groups[] = {NID_X25519, NID_secp256r1, NID_secp384r1};
if (!SSL_CTX_set1_groups(ctx, groups, 3)) {
    /* handle error */
}
/* Enforce TLS 1.3 only */
SSL_CTX_set_min_proto_version(ctx, TLS1_3_VERSION);
SSL_CTX_set_max_proto_version(ctx, TLS1_3_VERSION);

Run this once per SSL_CTX at startup. Requires OpenSSL 1.1.1 or later headers and library.

Validation and Verification

List Supported Suites in Your Build

openssl ciphers -s -v -tls1_3

Run on any host with the target OpenSSL installation. Expect the five standard suites. If ChaCha20‑Poly1305 is missing, your build may lack it (some older vendor builds).

Test a Handshake

openssl s_client -connect example.com:443 -tls1_3 -ciphersuites 'TLS_AES_256_GCM_SHA384'

Check the output for Cipher : TLS_AES_256_GCM_SHA384 to confirm negotiation. Replace host/port and suite as needed.

Verify Group Negotiation

openssl s_client -connect example.com:443 -tls1_3 -groups X25519:P-256

Inspect the Server Key Exchange line (or Key‑Exch in newer output) to see which group was selected.

FIPS Provider Status

openssl list -providers
openssl version -f

The first command should show fips loaded; the second prints the FIPS module version if active. The FIPS provider must be explicitly loaded via config or OSSL_PROVIDER_load() before creating SSL_CTX.

Unit Test Loopback

In your test suite, create a server and client SSL_CTX, set suites and groups, perform a loopback handshake (SSL_new, SSL_connect/SSL_accept), then assert SSL_get_cipher(ssl) returns the expected suite name and SSL_get_shared_group(ssl, 0) returns the expected group NID.

Performance Benchmarking

On your target hardware, run:

openssl speed -evp chacha20-poly1305
openssl speed -evp aes-256-gcm

Compare the throughput numbers. Choose the suite that gives higher throughput for your workload. This step is essential when deploying on heterogeneous fleets.

Limitations and Cautions

• OpenSSL 1.1.1 does not support the Ciphersuites config directive; you must use the API.
• Setting an empty cipher list via SSL_CTX_set_cipher_list(ctx, "") does not disable TLS 1.2. Use SSL_CTX_set_min_proto_version(ctx, TLS1_3_VERSION) to enforce TLS 1.3 only.
• ChaCha20‑Poly1305 requires OpenSSL built with ChaCha20/Poly1305 support (default since 1.1.0). Some vendor builds (e.g., older RHEL 8) may lack it if compiled without enable-ec_nistp_64_gcc_128.
• Session ticket keys must be rotated (SSL_CTX_set_tlsext_ticket_key_cb). OpenSSL 3.0’s SSL_CTX_set_num_tickets() helps with stateless tickets but does not solve cross‑process key synchronization; you must share the key callback or use a synchronized store.
• FIPS provider must be explicitly activated. Merely building with the FIPS module is insufficient.

Practical Checklist

1. Identify hardware capabilities (AES‑NI present?) and compliance requirements (FIPS?).
2. Select suites: prefer AES‑GCM for AES‑NI, ChaCha20‑Poly1305 otherwise; restrict to FIPS‑approved suites if required.
3. Choose group order: X25519 first, then NIST curves.
4. Apply configuration via openssl.cnf (OpenSSL 3.0+) or API (all versions).
5. Validate with openssl ciphers, s_client, and loopback unit tests.
6. Benchmark on target hardware to confirm performance assumptions.
7. Set up session ticket key rotation and, if multi‑process, a shared key store.

Following these steps gives you a deliberate, verifiable TLS 1.3 configuration that matches your security, performance, and compliance goals.