pfSense CARP Cluster Design: Failover Boundaries and Sync Limits

Problem and takeaway

You have two pfSense firewalls protecting one LAN. If the active firewall fails, clients lose their default gateway until someone manually reconfigures client routing or moves a cable. CARP (Common Address Redundancy Protocol) lets both nodes share a virtual IP address, so the backup node can answer ARP for the gateway and take over traffic flow without client changes.

The useful takeaway is narrow: CARP provides gateway failover, not load balancing, and state synchronization only works when both nodes run identical configuration over a trusted layer-2 path. This note covers the smallest viable design, where trust ends, how to verify operation, and which conditions should push you to a different architecture.

Requirements

• Two pfSense nodes with matching interface assignments and the same software release.
• A dedicated interface or VLAN between the nodes for configuration and state synchronization.
• A unique CARP VHID (Virtual Host Identifier) for each virtual IP on the shared layer-2 domain.
• Consistent system time, preferably via NTP, on both nodes.
• Physical or switched layer-2 connectivity for CARP advertisements; CARP does not route across ordinary Layer 3 boundaries.

Smallest suitable design

Use two nodes in active/backup mode. Each node connects to WAN, LAN, and a dedicated sync interface. The LAN carries a CARP virtual IP that clients use as their default gateway. Node A starts as MASTER with a lower advskew (advertisement skew); Node B starts as BACKUP. If Node A stops advertising, Node B assumes the virtual IP after the dead interval.

Element	Node A	Node B
WAN interface	igb0	igb0
LAN interface	igb1	igb1
Sync interface	igb2	igb2
LAN virtual IP	10.0.1.1/24	10.0.1.1/24
CARP VHID	1	1
Advskew	0	100

Replace igb0, igb1, igb2, and 10.0.1.1/24 with your actual interface names and subnet. The VHID must be unique on that broadcast domain; another cluster using VHID 1 on the same LAN will conflict.

Trust and data boundaries

• CARP boundary: CARP advertisements are link-layer multicast. Any device on the same VLAN can send CARP announcements, so the CARP-enabled LAN and sync VLANs must terminate on trusted switches. Do not stretch CARP across an untrusted WAN.
• Sync boundary: Configuration and state sync travel over the sync interface. If that link crosses anything other than a trusted, private network, carry it inside an IPsec tunnel or restrict sync to configuration only and accept state loss on failover.
• ARP/NDP boundary: The node holding the virtual IP answers ARP for it. Duplicate virtual IPs or overlapping VHIDs on the same layer-2 domain create ARP conflicts and can blackhole traffic without generating an obvious error on either firewall.

Operational checks

Run these checks after initial setup and after any rule or interface change.

1. Verify CARP state

Where: pfSense shell, via SSH or Diagnostics > Command. Required permissions: admin or root. This command is read-only.

carpctl show

Expected check: each CARP-enabled interface lists its VHID and state as MASTER or BACKUP. The node owning the virtual IP should show MASTER; the peer should show BACKUP. If both show MASTER, you have a split-brain condition, usually from duplicate VHIDs or a blocked sync path. Risk: none; the command only reads status.

2. Verify configuration consistency

Where: copy /conf/config.xml from each node, then compare locally. Required permissions: root to read the file, or use Diagnostics > Backup & Restore as admin.

diff pfsense-a-config.xml pfsense-b-config.xml

Expected check: no output means the configurations match. Differences indicate sync failure or a manual edit on one node. Risk: the configuration file contains secrets; transfer and store it securely, and delete local copies after comparison.

3. Verify clock alignment

Where: System > NTP or the shell command date on both nodes. Expected check: system time should agree within a few seconds. Drift can desynchronize state tables and cause established sessions to drop after failover.

4. Verify state synchronization

Where: Diagnostics > States on the MASTER node. Expected check: active sessions appear, and the system log does not report repeated state-sync errors. Under high throughput, confirm the sync link is not saturated before adding more concurrent connections.

Failure modes

• Clock drift: If node clocks diverge, state table entries may be considered stale on the backup, causing failover to drop active TCP sessions even though the virtual IP moves.
• VHID or virtual IP overlap: Two clusters using the same VHID on one LAN, or the same virtual IP assigned to unrelated nodes, cause conflicting ARP replies and intermittent traffic loss.
• Sync-link latency or congestion: State synchronization over a high-latency or congested link can delay failover beyond typical session timeout values, so connections break before the backup takes over cleanly.
• State-table saturation: At very high connection rates, synchronizing every state entry can become a bottleneck. Optimize sync intervals or reduce stateful inspection load; CARP itself still moves the IP, but existing sessions may not survive.

Conditions that change the design

• You need active-active load balancing: CARP is not a load balancer. Use a dedicated load-balancing tier or split client traffic by gateway rather than expecting both nodes to forward for the same virtual IP simultaneously.
• Nodes are separated by a Layer 3 network: CARP advertisements do not cross routers by default. Consider independent firewalls with routing-protocol failover instead of a stretched CARP cluster.
• State sync must cross a WAN: do not send raw state sync over the public internet. Use an IPsec-encrypted sync tunnel, sync configuration only, or accept that failover will drop existing state.

Configuration changes and rollback

Editing CARP, interfaces, or sync settings writes state. Before changing anything, download /conf/config.xml from both nodes through Diagnostics > Backup & Restore. If a change breaks failover, upload the known-good backup to the affected node and reapply the configuration. This restores the previous state; test CARP status again with carpctl show after restoring.

Practical validation

Schedule a maintenance window. Stop the CARP service or disable the LAN interface on the MASTER node and confirm the BACKUP node assumes the virtual IP and clients retain gateway reachability. Re-enable the MASTER and observe whether it resumes the MASTER role according to your preemption setting. Do not run this test during peak traffic, because failover can briefly interrupt flows even in a healthy cluster.

Limitations

CARP moves the gateway address; it does not synchronize application state on servers behind the firewall, and it does not increase outbound bandwidth. The checks above confirm that the cluster is configured consistently, not that it will survive every failure scenario. Treat any change to VHID, virtual IP, or sync credentials as a change to the trust boundary and re-run the operational checks afterward.