CARP and Virtual IPs in pfSense - HA Cluster Setup

CARP (Common Address Redundancy Protocol) is a network-layer failover protocol developed by the OpenBSD team as a patent-free alternative to Cisco HSRP and the VRRP standard. In pfSense, CARP provides automatic failover of virtual IP addresses between cluster nodes when the primary node fails. Network clients communicate with a virtual address (CARP VIP) that is always served by the active node - failover occurs transparently without requiring any network configuration changes on client devices.

Each cluster node periodically sends heartbeat messages on every interface where a CARP VIP is configured. At default frequency settings (base 1, skew 0 for master), a heartbeat is sent approximately once per second. The node with the lowest skew value sends heartbeats faster and assumes the master role. If the backup node stops receiving heartbeats within the expected interval, it promotes itself to master and begins handling traffic destined for the CARP VIP.

IP Addressing Requirements

A properly functioning CARP cluster requires careful IP address planning. Each network interface participating in failover needs a minimum of three IP addresses within the same subnet.

Addressing Scheme

The sync interface does not require a CARP VIP because it is used exclusively for direct communication between cluster nodes (pfsync and XMLRPC).

Subnet Size Requirements

The WAN interface requires at minimum a /29 subnet (6 usable addresses) to accommodate two node addresses, the CARP VIP, and the upstream gateway address. In production environments, a /28 or larger subnet is recommended to allow for future expansion or additional CARP VIPs.

The LAN interface typically has no subnet size constraints - a standard /24 subnet provides sufficient address space.

Warning:

The CARP VIP must reside in the same subnet as the individual node addresses on that interface. Using an address from a different subnet will render CARP non-functional on that interface.

Addressing for Additional Interfaces

When additional interfaces are present (DMZ, OPT1, OPT2, etc.), each requires the same three-address scheme. Document the addressing plan for all cluster interfaces in advance to prevent conflicts and simplify troubleshooting.

Hardware Preparation

Cluster reliability depends directly on the quality of hardware and network infrastructure preparation.

Node Hardware Requirements

Both cluster nodes must have identical network interface configurations. This is a critical requirement - pfSense performs configuration synchronization via XMLRPC based on interface assignment order. If the interface order differs between nodes, synchronization of firewall rules, NAT, and other settings will produce incorrect results.

Hardware recommendations:

• Identical network adapter models on both nodes to ensure consistent interface assignment order
• The same pfSense version on both nodes
• Sufficient RAM to store the state table (pfsync replicates all states)
• Separate power sources to eliminate a single point of failure

Warning:

Interfaces on both nodes must be assigned in exactly the same order. If on the primary node WAN is assigned to igb0, LAN to igb1, and Sync to igb2, the secondary node must have the exact same mapping. A mismatch will cause configuration synchronization to apply settings to the wrong interfaces.

Dedicated Sync Interface

A separate network interface must be dedicated to pfsync and XMLRPC communication between nodes. Using the LAN or WAN interface for synchronization is technically possible but strongly discouraged for the following reasons:

• State synchronization traffic can be substantial under heavy load
• pfsync transmits data in cleartext - interception on a shared segment presents a security risk
• A dedicated interface isolates synchronization from production traffic

The sync interface should be connected via a crossover cable directly between nodes or through a dedicated VLAN on the switch, isolated from general traffic.

Switch Configuration

The switch connecting both cluster nodes must support the following capabilities:

On managed switches, verify that port security, MAC address limiting, or DHCP snooping features do not interfere with CARP operation. These protection mechanisms may interpret the appearance of a new MAC address on a port or CARP multicast traffic as an anomaly.

In environments where multicast traffic is unavailable or blocked (certain cloud providers, specific switch configurations), pfSense Plus supports unicast CARP mode. In this mode, heartbeat messages are sent directly to the peer node’s IP address instead of the multicast group.

Creating Virtual IPs

CARP virtual IP addresses are created through the pfSense web interface at Firewall > Virtual IPs. Configuration is performed only on the primary node - with properly configured XMLRPC synchronization, virtual addresses are automatically replicated to the secondary node.

CARP VIP Parameters

When creating a new Virtual IP of type CARP, the following parameters must be specified:

Fig. 1. Creating a Virtual IP of type CARP

VHID Assignment

VHID (Virtual Host ID) is a unique identifier for a CARP virtual group within a single broadcast domain (VLAN or physical segment). VHID values range from 1 to 255.

Recommended VHID assignment scheme:

VHID values must not overlap within the same broadcast domain. If multiple independent CARP clusters exist in a segment (for example, two pfSense clusters on the same WAN segment), each must use unique VHIDs.

Warning:

A VHID conflict between independent clusters in the same broadcast domain causes unpredictable behavior - both clusters will compete for the same virtual group, resulting in continuous master/backup role flapping.

Advertising Frequency

The Advertising Frequency parameter controls CARP heartbeat timing and consists of two components:

• Base - base interval in seconds (default 1)
• Skew - additional delay in units of 1/256 second

The node with the lowest value of (base + skew/256) sends heartbeats faster and becomes master. The primary node should use skew 0, the secondary should use skew 100 or higher. A higher skew value means lower node priority.

When synchronization via XMLRPC is active, skew values are automatically adjusted on the secondary node to maintain the correct priority hierarchy.

Configuring CARP on the Primary Node

CARP configuration on the primary node is performed first. All settings are subsequently replicated to the secondary via XMLRPC.

Prerequisites

Before creating CARP VIPs on the primary node:

1. Assign all network interfaces via Interfaces > Interface Assignments
2. Configure static IP addresses on each interface (WAN, LAN, Sync)
3. Verify that the primary node has network access to the upstream gateway
4. Set a unique hostname for the primary node via System > General Setup

Creating the WAN CARP VIP

Navigate to Firewall > Virtual IPs and click Add.

Enter the following parameters:

Click Save, then Apply Changes.

Fig. 2. WAN CARP VIP parameters on the primary node

Creating the LAN CARP VIP

Repeat the procedure for the LAN interface:

Click Save, then Apply Changes.

Additional Interfaces

For each additional interface (DMZ, OPT1, OPT2), create a separate CARP VIP following the same procedure. Each VIP must have a unique VHID within its broadcast domain.

Example for a DMZ interface:

IPv6 CARP VIPs

In dual-stack environments, each IPv6 address requires a separate CARP VIP with its own VHID. The procedure is identical to IPv4, but the Address field contains an IPv6 address with the appropriate prefix length.

Configuring CARP on the Secondary Node

Secondary node configuration is performed after the primary setup is complete. When XMLRPC synchronization is functioning correctly, CARP VIPs are created automatically on the secondary node. If synchronization has not yet been configured, virtual addresses must be created manually.

Prerequisites

On the secondary node:

1. Assign interfaces in exactly the same order as on the primary
2. Configure unique static IP addresses on each interface (different from the primary)
3. Set a unique hostname via System > General Setup
4. Verify network connectivity between nodes via the sync interface

Warning:

Do not connect the secondary node to the LAN switch until unique IP addresses, different from the primary node’s addresses, have been configured. Connecting two nodes with identical addresses to the same segment will cause an IP address conflict and loss of connectivity.

Manual VIP Creation on the Secondary

If XMLRPC synchronization has not been configured yet, create CARP VIPs manually with identical parameters, changing only the Skew value:

A Skew value of 100 on the secondary ensures that during normal operation of both nodes, the primary always sends heartbeats sooner and retains the master role. A higher Skew value means lower priority in master elections.

Automatic VIP Synchronization

With XMLRPC synchronization configured, CARP VIPs are created on the secondary automatically. pfSense automatically adjusts the Skew value on the secondary node to maintain the correct priority hierarchy. In this case, manual VIP creation on the secondary is not required and not recommended - duplication may cause conflicts.

Verifying Status

After creating CARP VIPs on both nodes, verify that the cluster is operating correctly.

Viewing CARP Status

Navigate to Status > CARP (failover) on each node. The page displays the current state of each CARP VIP.

Fig. 3. CARP status - primary node in MASTER state

Expected cluster state:

If all VIPs on the primary show MASTER and all VIPs on the secondary show BACKUP, the cluster is functioning correctly.

Manual Failover Test

pfSense allows planned failover without physically disconnecting hardware. On the primary node’s Status > CARP (failover) page, click Enter Persistent CARP Maintenance Mode. This transitions all primary VIPs to BACKUP state, and the secondary assumes the MASTER role.

Testing procedure:

1. On the primary, navigate to Status > CARP (failover)
2. Click Enter Persistent CARP Maintenance Mode
3. Verify that all VIPs on the primary have transitioned to BACKUP
4. On the secondary, verify that all VIPs have transitioned to MASTER
5. Test service availability through the CARP VIPs
6. On the primary, click Leave Persistent CARP Maintenance Mode
7. Verify that the primary has resumed MASTER status for all VIPs

Command-Line Verification

CARP status can be checked via SSH or console:

ifconfig | grep -A 2 carp

Output on the master node:

carp: MASTER vhid 200 advbase 1 advskew 0
carp: MASTER vhid 1 advbase 1 advskew 0

Output on the backup node:

carp: BACKUP vhid 200 advbase 1 advskew 100
carp: BACKUP vhid 1 advbase 1 advskew 100

Using CARP VIPs

Creating CARP VIPs is a necessary but insufficient step. For an HA cluster to function correctly, all network services must be bound to CARP VIPs instead of individual interface addresses.

NAT Rules

All outbound NAT rules must use the CARP VIP as the Translation Address. When using the Interface Address, translation works only on whichever node currently holds the master role. After failover to the secondary, outbound NAT ceases to function because clients expect responses from the CARP VIP address, while the secondary would translate through its own address.

Configuration at Firewall > NAT, Outbound tab:

1. Switch the mode to Hybrid Outbound NAT rule generation
2. Create a rule:
   • Interface: WAN
   • Source: LAN Subnets (or a specific subnet)
   • Translation Address: select the CARP VIP (198.51.100.200)
3. Click Save, then Apply Changes

Fig. 4. Outbound NAT rule bound to CARP VIP

Warning:

Do not use a CARP VIP as the Source in outbound NAT rules for the firewall’s own traffic. This can break protocols that depend on the source address (DNS, NTP, package updates). Rules using CARP VIPs should apply only to traffic from internal subnets.

Port Forward Rules

Port Forward rules (Firewall > NAT > Port Forward) must specify the CARP VIP in the Destination field:

Using the WAN interface address instead of the CARP VIP will make port forwarding work on only one node.

Firewall Rules

Firewall rules for inbound traffic to CARP VIPs are created automatically when configuring Port Forward rules (if the automatic rule creation option is enabled). For manual rules, specify an alias or the specific CARP VIP as the Destination.

DHCP Server

The DHCP server configuration must use the CARP VIP as the gateway and DNS server distributed to clients:

Configuration at Services > DHCP Server, LAN tab:

This ensures that network clients communicate with the virtual address that is always served by the active cluster node. Specifying an individual node address causes clients to lose gateway and DNS access when that specific node fails.

VPN Tunnels

IPsec and OpenVPN tunnels should be bound to CARP VIPs:

• IPsec: in Phase 1 settings, specify the CARP VIP as the Interface or Local Address
• OpenVPN: in server or client settings, specify the CARP VIP in the Interface field

When using IPsec with a CARP VIP, create a separate outbound NAT rule for ISAKMP traffic (port 500/UDP) with the Static Port flag and bound to the CARP VIP.

Migration from Other Platforms

Administrators transitioning to pfSense from other firewall platforms can use the following mappings to plan their HA cluster migration.

Cisco ASA Failover

In Cisco ASA, a failover pair can trigger automatic switchover on both node failure and individual interface failure. In pfSense, CARP operates per-interface - if a node loses connectivity on one interface, it may transition to BACKUP only for that interface while remaining MASTER on others. This condition (split master) requires additional monitoring via Gateway Groups or custom scripts.

Fortinet FortiGate HA

FortiGate HA supports active-active clusters with load balancing. pfSense supports only active-passive. When migrating from an active-active FortiGate deployment, note that all traffic will be processed by a single pfSense node.

MikroTik VRRP

Key difference: in MikroTik VRRP, a high priority value (255) means highest priority, while in pfSense CARP, a low skew value (0) means highest priority. When planning migration, the priority logic must be inverted.

MikroTik does not have a built-in equivalent of XMLRPC for configuration synchronization between nodes. Administrators accustomed to manual configuration synchronization on MikroTik will benefit significantly from pfSense’s automatic synchronization via XMLRPC.

Troubleshooting

Both Nodes in MASTER State

The most common issue is both nodes simultaneously displaying MASTER status for the same VIPs. Possible causes:

Multicast traffic blocked. CARP uses multicast address 224.0.0.18 for heartbeat messages. If the switch or intermediate network equipment blocks multicast, nodes do not receive each other’s heartbeats and both consider themselves master.

Diagnosis:

tcpdump -i em0 proto carp

If heartbeats from the second node are not visible, the problem lies in L2 network connectivity.

Mismatched CARP passwords. If Virtual IP Password values differ between nodes, heartbeat messages are rejected as unauthenticated.

Resolution: verify that the CARP VIP password is identical on both nodes. When using XMLRPC synchronization, the password is replicated automatically.

VHID conflict. Another device in the same broadcast domain using the same VHID causes competition for the virtual group.

Resolution: change the VHID to a unique value not used by any other device in the segment.

Split Brain

A split brain condition occurs when both nodes are master for different sets of interfaces. For example, the primary is MASTER on WAN but BACKUP on LAN, while the secondary has the opposite state. This results in asymmetric routing and connectivity loss.

Causes:

• Physical failure of a network interface on one node
• Cable or switch port problem on a specific interface
• Different VLAN configurations on switch ports

Diagnosis:

1. Check the status of all interfaces on both nodes via Status > CARP (failover)
2. Verify physical connectivity for each interface via Status > Interfaces
3. Review CARP logs: Status > System Logs, System tab, filter by “carp”

Resolution: address the root cause of connectivity loss on the specific interface. If immediate resolution is not possible, temporarily place the problematic node in maintenance mode to ensure routing integrity.

CARP Not Failing Over

If the secondary does not assume the master role when the primary fails:

Check heartbeats. On the secondary, run:

tcpdump -i em0 proto carp

If heartbeats from the primary are still arriving, the primary has not actually failed - the problem lies elsewhere.

Check Skew value. If the skew on the secondary is excessively high (for example, 240), the transition may take an extended period. Reduce the skew to 100.

Check the network stack. Verify that CARP traffic is not blocked by firewall rules on the corresponding interface of the secondary.

Check the demotion counter. pfSense may automatically increment the demotion counter when problems are detected, preventing the transition to MASTER:

sysctl net.inet.carp.demotion

A value of 0 is normal. A non-zero value indicates a condition preventing master role assumption.

Multicast Issues

In virtualized environments (VMware, Proxmox, Hyper-V), CARP multicast traffic may be blocked at the virtual switch level.

VMware vSwitch/vDS:

• Set Promiscuous Mode: Accept
• Set Forged Transmits: Accept
• Set MAC Address Changes: Accept

Proxmox (Linux Bridge):

• Verify that the bridge does not filter multicast
• Check the multicast_snooping parameter on the bridge

Hyper-V:

• Enable MAC Address Spoofing on the virtual adapter
• Allow multicast in virtual switch settings

Slow Failover

If failover takes longer than 5-10 seconds when the primary fails:

• Reduce advbase to 1 (if not already set)
• Reduce skew on the secondary (100 is recommended)
• Check CPU load on both nodes - under heavy load, CARP heartbeat processing may be delayed
• Verify that pfsync is correctly synchronizing states - without synchronization, the secondary must re-establish all connections, increasing recovery time

CARP Event Logging

To track failover events and aid in diagnostics, enable extended CARP logging:

Status > System Logs > Settings:

• Log CARP State Changes: enable

CARP events are recorded in the system log and available at Status > System Logs, System tab. When integrated with an external SIEM (such as Wazuh), CARP events can be used to create alerting rules for cluster failover notifications.

Related Sections

• Configuration Synchronization - XMLRPC and pfsync setup for full cluster operation
• Failover Scenarios - failover testing and outage planning
• Firewall Rules - configuring rules for the sync interface
• Outbound NAT - binding NAT rules to CARP VIPs
• pfSense VPN - binding VPN tunnels to CARP VIPs for failover