Troubleshooting CARP Failover When Secondary Stays in BACKUP State on pfSense

Recognizable Condition

The secondary pfSense unit remains in the BACKUP state while the primary is powered off, disconnected, or otherwise unreachable. The CARP virtual IP (VIP) is not answered by either node, causing loss of the HA address.

Cause / Diagnostic Overview

Possible Cause	What to Look For
CARP traffic blocked by firewall rules	Logs show dropped protocol 112 packets
Interface link down or missing IP	ifconfig shows no carrier or missing IP address
Mismatched VHID or password	CARP status shows authentication errors or different VHID
pfsync misconfiguration	pfsync interface errors or no SYNC state
Incorrect advskew/advbase	Secondary has equal or lower advskew than primary

Ordered Checks

1. View CARP status: ifconfig carp0 or Status → CARP in the web GUI. Look for state BACKUP and note advskew values.
2. Verify the physical interface: ifconfig igb0 (replace igb0 with your CARP‑enabled interface). Ensure status: active and an IP address is present.
3. Check firewall logs for blocked CARP: grep carp /var/log/filter.log or Diagnostics → System Logs → Firewall. Look for protocol 112 drops.
4. Confirm VHID and shared secret: System → High Avail. Sync → CARP Settings on both nodes. The VHID number and Password must be identical.
5. Inspect pfsync: ifconfig pfsync0 (or the interface you use for pfsync). Look for status: active and peerfsync messages. In the GUI, Status → pfSync shows SYNC.

Fixes Tied to Findings

• Allow CARP traffic: Add a firewall rule on each interface (LAN, WAN, SYNC, etc.) that passes protocol 112. Example CLI snippet:

  pass in quick on $LAN proto carp from any to any label \"Allow CARP\"
  pass out quick on $LAN proto carp from any to any label \"Allow CARP\"
  

  Apply via Firewall → Rules, select the interface, Action Pass, Protocol CARP, Source any, Destination any, then Save and Apply Changes.
• Repair link: Replace faulty NIC, cable, or SFP module. After fixing, run ifconfig igb0 and verify media: Ethernet autoselect (1000baseT ) and status: active.
• Align VHID and password: On both firewalls, set the same VHID (e.g., 1) and identical shared secret under CARP Virtual IP settings. Changing these will cause a brief state flip; perform during a maintenance window.
• Re‑initialize pfsync: If the pfsync interface shows errors, disable then re‑enable it: System → High Avail. Sync → pfsync Settings → Disable, Save, then Enable again. Or via CLI: ifconfig pfsync0 down followed by ifconfig pfsync0 up.
• Adjust advskew/advbase: Ensure the primary has a lower advskew (higher priority). For example, set Primary advskew = 0, Secondary advskew = 100. advbase should be identical (default 1). Change under CARP Virtual IP → Advanced.

Escalation Criteria

• If the secondary still shows BACKUP after applying the above fixes, check system logs for repeated CARP or pfsync errors (grep carp /var/log/system.log).
• Verify hardware health: check NIC LEDs, replace the unit, or test with a spare firewall.
• Consider updating to the latest stable pfSense version (e.g., 2.7.x) if a known bug is suspected.
• Gather a configuration backup (/conf/config.xml) and recent logs, then open a support ticket with Netgate.

Verification

1. Force a failover: either power off the primary firewall or disable its CARP interface (ifconfig carp0 down).
2. Watch the secondary: ifconfig carp0 should now show state MASTER and the VHID.
3. From an external host, ping the CARP VIP (e.g., ping 10.0.0.1) and confirm continuous replies.
4. Check that pfsync reports SYNC and that a configuration change made on the new primary replicates to the former secondary.

Limitations

• This guide assumes CARP is configured with a single VIP. Multi‑VIP setups require repeating the checks per VHID.
• If the primary is suffering a catastrophic hardware failure (e.g., motherboard), the secondary may never become MASTER until the primary is fully removed from the network.
• Virtualized environments may need to ensure promiscuous mode is enabled on the virtual switch for CARP traffic.

Rollback (if you changed firewall rules or CARP settings)

1. To revert a firewall rule, delete the rule you added or set its action to Block, then Apply Changes.
2. To revert VHID or password changes, restore the previous values from your configuration backup or re‑enter the original shared secret and VHID.
3. After rollback, repeat the verification steps to confirm the cluster returns to its original state.