Latency spikes in pfSense under high concurrent TCP connection rates

Neither state-table hash bucket contention nor QoS shaper rule evaluation is universally primary. Under steady-state high concurrency with many shaper rules, QoS rule evaluation is the more likely latency driver because state lookup is hash-based and average O(1), while shaper rule matching is sequential and O(n) in the number of rules. Under bursty connection establishment—many short-lived TCP flows or SYN-heavy traffic—state-table lock contention and state insertion are more likely to dominate, especially on pfSense 2.6.x or when the state table is undersized. pfSense 2.7.x, with its newer FreeBSD base, generally improves lock granularity and multi-core scaling, reducing but not eliminating state-table contention.

Confirmed facts vs likely explanations

Confirmed: state table lookups are hash-based on average; shaper rule evaluation is sequential; state table maximum and shaper rule count are independent configuration knobs; state table exhaustion causes drops, not just added latency; hardware offload can distort latency measurements.

Likely: lock contention often appears as a single CPU core saturated in kernel time; shaper latency scales with rule count, match complexity, and per-flow queue usage; 2.6.x is more prone to lock contention than 2.7.x under the same load.

Uncertain: the exact concurrency threshold where one factor overtakes the other; whether your workload is bursty or steady-state; version-specific lock granularity details. Treat these as items to verify on the actual system.

Diagnostic steps for this case

1. Record a baseline under load: current state count, maximum states, per-core CPU, and interrupt distribution.
2. Correlate latency spikes with either new-connection rate or sustained packet rate on established flows.
3. Temporarily disable the QoS shaper under controlled load. If spikes disappear, shaper evaluation is dominant. If they persist, focus on state-table locking, state limits, or another kernel path.
4. If shaper-dominated, reduce rule count, place most-matched rules first, simplify match criteria, and avoid unnecessary per-flow queues.
5. If state-table-dominated, raise the maximum states only if current usage approaches the limit; that prevents exhaustion but does not directly fix hash bucket lock contention. Consider 2.7.x if still on 2.6.x.

Verification commands and what to look for

Command	Check	Interpretation
pfctl -si	State table searches, inserts, current entries	High insert/search rate with spikes suggests connection churn and possible lock pressure.
pfctl -sm	State limits	Current entries near maximum means exhaustion risk, not just latency.
top -H	Per-thread CPU	One core saturated in kernel time points to lock contention or single-threaded packet processing.
vmstat -i	Interrupt distribution	Uneven NIC interrupt handling can mimic or mask state-table contention.
pfctl -sr and dnctl list	Shaper rules and dummynet objects	Many rules or queues increase per-packet evaluation cost.
netstat -s -p tcp	TCP connection counters	Correlate new connection rate with latency spikes.

One missing detail that changes the recommendation

Are the latency spikes correlated with new-connection rate (SYN/s) or with sustained packet rate on already-established flows? If SYN-correlated, prioritize state-table sizing, state insertion cost, and lock contention. If steady-state, prioritize shaper rule count, rule order, and per-flow queue complexity.

Caveats

Disable LRO/TSO for accurate latency testing; otherwise offload can hide or amplify the symptom. State table exhaustion drops packets rather than merely adding latency. Exact thresholds and lock behavior vary between pfSense 2.6.x and 2.7.x, so confirm on the target version before changing production settings.