Enabling and Validating TCP Fast Open on Linux Servers

Decision and Constraints

Enable TCP Fast Open (TFO) when a service handles many short‑lived TCP connections (e.g., HTTP, gRPC) and both client and server run Linux kernel 3.7 or newer. The change requires root privileges to modify a sysctl variable, and applications must either set the TCP_FASTOPEN socket option or rely on the global setting.

Supported Options

Value	Meaning	Direction
0	Disabled	None
1	Client‑only	Client may send data in SYN
2	Server‑only	Server may accept data in SYN
3	Both client and server	Both sides may use TFO

Trade‑offs

• Reduces one round‑trip, lowering latency for short flows.
• Introduces a small replay‑attack window if spoofed source addresses are accepted.
• Middleboxes that strip the TFO option can cause fallback to a normal three‑way handshake or connection reset.
• Using server‑only (value 2) limits exposure while still allowing clients that initiate TFO to benefit.

Implementation

1. Create a sysctl configuration file as root:

   # /etc/sysctl.d/99-tcp-fastopen.conf
   net.ipv4.tcp_fastopen=3

2. Apply the setting:

   sysctl --system

3. Verify the value took effect:

   cat /proc/sys/net/ipv4/tcp_fastopen

   Expected output: 3.

Validation

1. Confirm the sysctl value as shown above.
2. Capture SYN packets to see if they carry data:

   tcpdump -i any -s 0 -nn -vv port 80 and 'tcp[tcpflags] == tcp-syn'

   Look for packets where the payload length exceeds the TCP header (typically >20 bytes), indicating a SYN+data segment.
3. Test with a client that sets the socket option, for example using curl:

   curl --tcp-fastopen http://localhost/

   Alternatively, a minimal C program can call setsockopt(fd, IPPROTO_TCP, TCP_FASTOPEN, &enable, sizeof(enable)) and then inspect TCP_INFO after connect(); the tcpi_options field should have the TCP_FASTOPEN bit set.

Limitations and Practical Checks

• Ensure any load balancers, firewalls, or middleboxes in the path preserve the TFO option; otherwise connections fall back to a normal handshake.
• Monitor for increased SYN‑packet size due to the TFO cookie on networks with strict MTU limits; fragmentation can cause performance degradation or drops.
• Check for possible replay‑attack exposure; consider enabling net.ipv4.tcp_syncookies and using server‑only mode if the risk is a concern.
• After applying the setting, re‑run the validation steps to confirm the change persists across reboots.