pfSense Configuration Sync - pfsync and XMLRPC

Configuration and state synchronization is the foundation of a functional pfSense HA cluster. Without synchronization, the standby node lacks current firewall rules and connection state information, resulting in dropped sessions during failover. pfSense employs two independent synchronization mechanisms: pfsync for state table replication and XMLRPC for configuration data. Each serves a distinct purpose and requires separate configuration.

State Table Synchronization (pfsync)

pfsync is a FreeBSD kernel-level protocol that replicates the pf firewall state table between cluster nodes. The state table contains records for every active connection traversing the firewall: TCP sessions, UDP flows, ICMP exchanges, and NAT translations.

Whenever a state entry is created, modified, or removed on one node, pfsync transmits the update to the peer node. This ensures that when failover occurs, existing sessions survive the transition - network clients experience no connection interruptions.

What pfsync Synchronizes

What pfsync Does NOT Synchronize

pfsync is exclusively concerned with the state table. The following data falls outside its scope:

• Firewall and NAT rules (handled by XMLRPC)
• Interface configuration
• Routes and ARP tables
• Traffic shaper queues
• pf table contents (aliases, bogons, snort blocklists)

How It Works

pfsync operates at Layer 2, using IP protocol number 240 (PFSYNC). By default, updates are sent to multicast address 224.0.0.240 on the designated sync interface. When a dedicated sync link connects two nodes, multicast traffic remains confined to that segment.

Updates are sent in batch mode to reduce overhead. pfSense groups multiple state changes into a single pfsync packet, dispatched at intervals governed by the Defer updates setting. When deferred updates are enabled (the default), the system briefly waits before sending, allowing multiple changes to be combined into one packet. This reduces sync interface load at the cost of minimal synchronization latency.

Configuring pfsync

pfsync configuration is performed on the primary node via System > High Avail. Sync under the State Synchronization Settings (pfsync) section.

If the pfsync Synchronize Peer IP field is left empty, pfsync sends updates to the multicast address. Specifying an explicit IP address switches pfsync to unicast mode, which is recommended for production clusters.

Warning:

The secondary node requires its own pfsync configuration with the primary node’s IP address (e.g., 172.16.1.2) in the Peer IP field. Unlike XMLRPC, pfsync must be configured on both cluster nodes.

Bandwidth Estimation

pfsync traffic volume depends on the rate of connection creation and modification. Each state update consumes approximately 300-400 bytes. Use the following guidelines to estimate sync interface bandwidth requirements:

A typical installation handling 10,000 active connections and 500 new connections per second generates approximately 200-300 Kbps of pfsync traffic on the sync interface. High-throughput environments with tens of thousands of new connections per second may produce several megabits per second.

A minimum 1 Gbps link is recommended for the sync interface. In environments with exceptionally high load (over 50,000 connections per second), consider a 10 Gbps interface or separating pfsync and XMLRPC onto distinct physical interfaces.

Configuration Synchronization (XMLRPC)

XMLRPC (XML Remote Procedure Call) is the mechanism pfSense uses to replicate configuration from the primary node to the secondary. Unlike pfsync, XMLRPC operates at the application layer and transfers structured configuration data - firewall rules, NAT, VPN tunnels, aliases, and other settings - rather than connection states.

Synchronization Direction

XMLRPC operates strictly one-way: from primary to secondary. All configuration changes must be made exclusively on the primary node. When properly configured, every change saved on the primary automatically triggers transmission of the updated configuration section to the secondary.

Warning:

Changes made directly on the secondary node will be overwritten during the next synchronization from the primary. The only exceptions are parameters explicitly excluded from synchronization (hostname, interface IPs, CARP skew values).

Configuring XMLRPC

XMLRPC configuration is performed only on the primary node via System > High Avail. Sync under the Configuration Synchronization Settings (XMLRPC Sync) section.

XMLRPC uses HTTPS (TCP 443) for data transmission. The sync interface on the secondary node must permit inbound traffic on port 443 from the primary node’s IP address.

Synchronization Areas

Below the connection parameters are checkboxes that control which configuration sections are replicated. All areas are enabled by default. The recommended practice is to leave all checkboxes active unless a specific section requires independent configuration on each node.

What Should NOT Be Synchronized

Certain parameters must remain unique to each cluster node. XMLRPC automatically excludes the following from synchronization:

• Hostname and domain - each node requires a unique name for identification in the cluster and in log files
• Interface IP addresses - individual addresses on each interface are unique per node
• CARP skew settings - skew values are automatically adjusted to maintain the correct priority hierarchy
• pfsync parameters - pfsync configuration (Peer IP) differs on each node

Additional sections can be manually excluded from synchronization. A common example is the DHCP Server, when address ranges on the primary and secondary differ (e.g., when splitting the address pool between nodes to prevent conflicts during a split-brain scenario).

DHCP Synchronization Considerations

When DHCP server synchronization is enabled, both nodes receive identical range configurations. Under normal operation, only the master node’s DHCP server is active because clients address their requests to the CARP VIP serving as the default gateway. However, in a split-brain scenario (both nodes claim master status), two DHCP servers with identical ranges may assign the same address to different clients.

To prevent split-brain address conflicts, either manually divide the DHCP pool between nodes or use DHCP Failover (ISC DHCP failover peer) if the pfSense version supports it.

Dedicated Sync Interface

Both pfsync and XMLRPC use the sync interface for inter-node communication. Dedicating a separate physical or virtual interface to synchronization traffic is a mandatory recommendation for production clusters.

Sync Interface Requirements

Why LAN or WAN Should Not Be Used

Using the LAN or WAN interface for synchronization introduces several risks:

1. Security - pfsync transmits data in cleartext, including state table contents. On a shared segment, this data is exposed to interception.
2. Performance - synchronization traffic competes with user traffic, potentially causing sync delays under heavy load.
3. Reliability - problems on the LAN/WAN interface (congestion, switch failure) simultaneously disrupt both synchronization and traffic handling.

Sync Interface Configuration

On both cluster nodes:

1. Assign the physical interface for synchronization via Interfaces > Interface Assignments (if not already assigned)
2. Navigate to Interfaces > Sync and set a static IP address:
   • Primary: 172.16.1.2/24
   • Secondary: 172.16.1.3/24
3. Enable the interface (Enable Interface)
4. Do not specify a gateway - the sync interface must not have a default gateway

Sync Interface Firewall Rules

Firewall rules must be created on the sync interface to permit synchronization traffic. The minimum rule set:

A simpler approach is to create a single rule allowing all traffic on the sync interface between node addresses. Since the sync interface is isolated from other networks, this does not introduce additional security risks.

Verifying Synchronization

Verifying pfsync

To confirm pfsync is working, verify that the state tables on both nodes contain matching entries.

On the primary node via Diagnostics > States:

• Note the current number of active states
• Initiate a new connection (e.g., open a web page through the firewall)
• Verify the new state appears in the table

On the secondary node, perform the same check:

• Navigate to Diagnostics > States
• Confirm the state count approximately matches the primary
• Locate the same connection entry that was created on the primary

The command line can also be used:

pfctl -s info | grep "current entries"

Values on both nodes should be close (minor discrepancies of a few dozen entries are expected due to synchronization latency).

Verifying XMLRPC

To verify XMLRPC synchronization:

1. On the primary node, create a test alias via Firewall > Aliases (e.g., test_sync_alias with a single IP address)
2. Click Save and Apply Changes
3. On the secondary node, navigate to Firewall > Aliases and confirm the test alias has appeared
4. After verification, delete the test alias on the primary - it should also be removed from the secondary

The system log at Status > System Logs > System > General on the secondary node should contain entries confirming successful configuration receipt from the primary.

Monitoring Sync Status

CARP and synchronization status can be checked via Status > CARP (failover). This page displays:

• Current status of each CARP VIP (MASTER/BACKUP)
• Time of last synchronization
• Synchronization errors (if any)

Troubleshooting Synchronization Issues

XMLRPC Sync Fails

Symptom: changes made on the primary do not appear on the secondary.

Possible causes and solutions:

Partial Synchronization

Symptom: some configuration areas synchronize while others do not.

This is typically caused by disabled checkboxes in the XMLRPC Sync Settings section. Verify that all required areas are checked on the primary node.

Partial synchronization can also occur when configuration conflicts exist. For example, if a record with the same identifier has been manually created on the secondary, XMLRPC may fail to overwrite it. In such cases, delete the conflicting entry on the secondary and re-trigger synchronization.

To force a full re-synchronization, navigate to System > High Avail. Sync on the primary and click Save without making changes - this triggers retransmission of all checked areas.

Certificate Issues

Symptom: XMLRPC synchronization fails with an SSL/TLS error.

XMLRPC connects to the secondary node over HTTPS. If the self-signed web interface certificate on the secondary has expired or is corrupted, synchronization will fail.

Resolution:

1. On the secondary node, navigate to System > Cert. Manager > Certificates
2. Locate the webConfigurator default certificate (or equivalent)
3. Check the validity period - if expired, delete the certificate
4. Navigate to System > Advanced > Admin Access and generate a new self-signed certificate
5. On the primary, re-save the XMLRPC settings to trigger synchronization

pfsync Not Working

Symptom: the state table on the secondary is empty or significantly differs from the primary.

High Synchronization Latency

Symptom: states on the secondary appear with noticeable delay (seconds).

When Defer updates is enabled, pfsync accumulates changes before sending. If latency is critical (e.g., for clusters with aggressive failover timing), deferred updates can be disabled. However, this increases load on the sync interface and CPU on both nodes.

Sync interface congestion can also cause latency. If pfsync traffic exceeds the interface bandwidth, updates will be dropped. In this case, increase the sync link speed or optimize the number of tracked states.

Related Sections

• CARP and Virtual IPs - CARP VIP configuration, addressing requirements, and cluster hardware preparation
• Failover Scenarios - failover testing, maintenance planning, and cluster monitoring
• Firewall Rules - configuring sync interface rules for XMLRPC and pfsync traffic
• pfSense VPN - IPsec and OpenVPN synchronization considerations in HA clusters