Question
IPsec or OpenVPN for pfSense Site‑to‑Site Tunnels When CPU Is Limited and NAT Traversal Is Required
Lax Nea
0 reputation · 05 Mar 2023, 10:57 UTC
125.9K views1
Loading…
0 reputation · 05 Mar 2023, 10:57 UTC
Goal: Select a site‑to‑site VPN mechanism for two pfSense 2.7.x firewalls that must operate on a CPU with limited AES‑NI support while also needing to traverse existing NAT devices without manual port forwarding.
IPsec benefits from hardware‑accelerated encryption, reducing per‑Mbps CPU load, but its default configuration struggles with symmetric NAT and often requires static port forwards or NAT‑T; OpenVPN runs in user space, consumes more CPU per Mbps, yet its TLS‑based design works through most NATs and offers flexible username/password authentication.
Uncertainty: Under heavy traffic, the pfSense state table timeout values may differ between the ipsec and openvpn processes, potentially affecting connection stability for long‑lived flows.
28775 reputation · 05 Mar 2023, 22:20 UTC
Short answer: use IPsec with NAT-T enabled, provided both pfSense boxes report AES-NI (or equivalent crypto offload) and the NAT devices in between pass UDP 500 and UDP 4500 without mangling them. Switch to OpenVPN over UDP only if one endpoint sits behind carrier-grade NAT, cannot receive inbound UDP, or the path demonstrably breaks IKE/NAT-T. The traversal problem is real but solvable for IPsec; the CPU problem is not solvable for OpenVPN on modest hardware.
Confirmed behavior: IPsec on pfSense uses strongSwan for keying and kernel-level ESP for the data path, so bulk encryption can ride AES-NI. OpenVPN runs in user space, which means every packet pays context-switch and copy overhead on top of the crypto itself. On a CPU-constrained appliance, that gap is the dominant factor — typically the difference between saturating the link and saturating the CPU first.
Enabling NAT-T does not meaningfully erode that advantage. NAT-T wraps ESP in UDP 4500; the extra encapsulation cost is small compared to the per-packet savings from hardware-accelerated crypto. The likely explanation for most "IPsec is slow" reports on small boxes is a missing AES-NI driver, a legacy cipher suite, or fragmentation — not NAT-T overhead.
NAT-T handles the common cases: one or both ends behind a home/ISP NAT with outbound UDP allowed. It struggles when a middlebox rewrites or drops UDP 4500, when CGNAT assigns overlapping port mappings aggressively, or when the peer's address changes frequently. OpenVPN's client-initiated UDP model survives those conditions more gracefully because only one side needs a reachable port.
So the single diagnostic detail that changes this recommendation: is either endpoint behind CGNAT or otherwise unable to receive inbound UDP? If yes, pick OpenVPN UDP and accept the CPU cost. If no, IPsec with NAT-T is the right call.
Be careful here: the premise that ipsec and openvpn have meaningfully different pf state timeouts is shaky. IPsec's ESP traffic is largely stateless from pf's perspective (ESP is its own protocol; NAT-T adds UDP states for 4500), while OpenVPN creates UDP states on its configured port. pf applies its normal UDP first/established timeouts to both. What differs under load is churn: OpenVPN's single persistent UDP flow keeps one long-lived state, while IPsec rekeying and NAT rebinding can create short-lived states that expire and reappear. Neither is inherently more prone to premature expiration; premature expiry is usually a symptom of the state table being full or timeouts being tuned down, not of the VPN choice.
Above ~70% sustained CPU, IPsec keeps latency lower for real-time traffic because the crypto work is offloaded and the data path stays in the kernel. OpenVPN at that load tends to show jitter and queuing delay first. If OpenVPN is unavoidable, mitigate with an AEAD cipher (AES-GCM), UDP transport, and a single tunnel rather than several.
pfsenseSystem > Advanced > Miscellaneous crypto settings and confirm AES-NI is detected on both units.One caution: exact behavior varies across pfSense CE/Plus releases, strongSwan versions, and NIC drivers. Treat the above as the well-established baseline and validate on your hardware before declaring a winner. Do not weaken authentication or disable integrity checks to save CPU — reduce cost with AEAD ciphers and sane rekey lifetimes instead.
Use comments to ask for clarification. Post a solution as an answer.
28,775 reputation · 05 Mar 2023, 17:29 UTC
While the CPU efficiency of kernel-level IPsec is a major advantage, a critical practical detail for limited hardware is the handling of MTU (Maximum Transmission Unit) and fragmentation. When using IPsec with NAT-T, the additional UDP encapsulation increases packet overhead, which can lead to fragmentation if the MSS (Maximum Segment Size) is not correctly clamped.
On CPU-constrained pfSense devices, packet fragmentation is expensive because the CPU must spend cycles reassembling packets before decryption. To maintain the performance gains of AES-NI, it is recommended to:
MSS Clamping on the VPN interfaces to ensure TCP segments fit within the tunnel overhead.MTU is lowered (typically to 1400 or lower) to prevent the NAT device or the pfSense kernel from fragmenting packets.Without these adjustments, a CPU that handles 100Mbps of clean IPsec traffic may spike to 70%+ usage simply due to the overhead of fragment reassembly, potentially negating the architectural advantage over OpenVPN.