# VXLAN Interfaces in VyOS

> Configure VXLAN in VyOS for overlay networks - L2-over-L3 tunnels, multicast and unicast flooding, BGP EVPN control plane for Yandex Cloud and VK Cloud

Source: https://opennix.org/en/docs/vyos/interfaces/vyos-vxlan/


VXLAN (Virtual Extensible LAN) is a network virtualization technology for building overlay networks on top of an existing IP infrastructure.

## Overview

VXLAN solves the scaling problems of traditional VLAN networks:
- Encapsulates Layer 2 Ethernet frames into Layer 4 UDP datagrams
- Uses a 24-bit VNI (VXLAN Network Identifier) instead of a 12-bit VLAN ID
- Supports up to 16 million logical networks (compared to 4096 VLANs)
- Operates over an existing IP network (underlay)
- Lets you stretch L2 segments across L3 boundaries

**Key components**:
- **VNI** (VXLAN Network Identifier) - the virtual network identifier (24 bits)
- **VTEP** (VXLAN Tunnel Endpoint) - the encapsulation/decapsulation point
- **Underlay network** - the physical IP network that carries the encapsulated traffic
- **Overlay network** - the virtual L2 network on top of the underlay

**UDP port**:
- VyOS default: **8472**
- IANA standard: **4789**

**Use cases**:
- Network virtualization in cloud providers
- Multi-tenant environments
- Data center interconnect (DCI)
- Linking geographically distributed L2 domains
- Kubernetes/Docker overlay networks

## Basic Configuration

### Creating a VXLAN Interface

```
set interfaces vxlan vxlan100 vni 100
commit
```

Minimal configuration - only the VNI (network identifier).

### Assigning an IP Address

```
set interfaces vxlan vxlan100 address 10.0.100.1/24
commit
```

### Interface Description

```
set interfaces vxlan vxlan100 description 'VXLAN Overlay Network 100'
commit
```

### MTU

It is recommended to reduce the MTU because of the encapsulation overhead:

```
set interfaces vxlan vxlan100 mtu 1450
commit
```

VXLAN adds 50 bytes of overhead:
- 20 bytes outer IP header
- 8 bytes UDP header
- 8 bytes VXLAN header
- 14 bytes outer Ethernet header

For the standard MTU of 1500:
- VXLAN MTU = 1500 - 50 = 1450

For jumbo frames (9000):
- VXLAN MTU = 9000 - 50 = 8950

## VXLAN Operating Modes

### 1. Multicast VXLAN

Uses multicast groups for automatic VTEP discovery and for flooding BUM traffic (Broadcast, Unknown unicast, Multicast).

**Configuration**:

```
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-interface eth0
set interfaces vxlan vxlan100 mtu 1450
set interfaces vxlan vxlan100 group 239.0.0.100
commit
```

**Parameters**:
- `vni` - the VXLAN network identifier
- `source-interface` - the interface for underlay traffic
- `group` - the multicast group for this VNI
- `mtu` - the maximum packet size

**How it works**:
1. BUM traffic is sent to the multicast group
2. All VTEPs in the group receive the traffic
3. VTEPs learn MAC addresses and build the FDB (Forwarding Database)
4. Unicast traffic goes directly between VTEPs

**Requirements**:
- The underlay network must support multicast (PIM)
- Not all cloud providers support multicast

### 2. Unicast VXLAN (Point-to-Point)

Explicitly specifies the remote VTEP by IP address. Does not require multicast.

**Configuration**:

```
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-interface eth0
set interfaces vxlan vxlan100 remote 203.0.113.10
set interfaces vxlan vxlan100 mtu 1450
commit
```

**Parameters**:
- `remote` - the IP address of the remote VTEP

**How it works**:
- All traffic is sent to the specified remote IP
- Suitable for point-to-point connections
- Does not scale for a large number of VTEPs

**Advantages**:
- Works in networks without multicast (most clouds)
- Simple configuration for a small number of nodes
- Deterministic behavior

**Disadvantages**:
- Does not support automatic VTEP discovery
- Requires manual configuration for each VTEP pair
- Scales poorly

### 3. Unicast VXLAN with Multiple Remote Endpoints

To connect to multiple VTEPs, use an FDB (Forwarding Database) or a bridge with FDB entries.

```
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-interface eth0

set interfaces bridge br100
set interfaces bridge br100 member interface vxlan100

# Static FDB entries (MAC to VTEP mapping)
# Syntax depends on the VyOS version
# Alternatively, use BGP EVPN for automatic distribution
commit
```

### 4. VXLAN via source-address

Instead of source-interface, you can specify the source IP directly:

```
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-address 198.51.100.5
set interfaces vxlan vxlan100 remote 203.0.113.10
commit
```

Useful when you have multiple interfaces or a loopback address.

## VLAN-to-VNI Mapping (Single VXLAN Device)

A Single VXLAN Device (SVD) lets you use one VXLAN interface for multiple VNIs by mapping VLANs to VNIs.

### SVD Configuration

```
# VXLAN interface with a base VNI
set interfaces vxlan vxlan0 vni 10000
set interfaces vxlan vxlan0 source-interface eth0

# Bridge for VLAN management
set interfaces bridge br0
set interfaces bridge br0 member interface vxlan0
set interfaces bridge br0 enable-vlan

# VLAN-to-VNI mapping
set interfaces bridge br0 member interface vxlan0 vlan-to-vni 10 vni 10010
set interfaces bridge br0 member interface vxlan0 vlan-to-vni 20 vni 10020
set interfaces bridge br0 member interface vxlan0 vlan-to-vni 30 vni 10030

# Physical interface for local VLANs
set interfaces ethernet eth1 vif 10
set interfaces ethernet eth1 vif 20
set interfaces ethernet eth1 vif 30

set interfaces bridge br0 member interface eth1.10 allowed-vlan 10
set interfaces bridge br0 member interface eth1.20 allowed-vlan 20
set interfaces bridge br0 member interface eth1.30 allowed-vlan 30

commit
```

**How it works**:
- VLAN 10 is locally mapped to VNI 10010
- VLAN 20 is locally mapped to VNI 10020
- VLAN 30 is locally mapped to VNI 10030
- A single VXLAN interface serves all VNIs
- The bridge performs VLAN-aware switching

**Advantages**:
- Scales to thousands of VNIs on a single interface
- Simplified configuration
- Lower system resource usage

## Configuration Examples

### Point-to-Point VXLAN (Yandex Cloud)

Connecting two VyOS instances in different Yandex Cloud subnets:

**Node 1** (192.0.2.10):
```
# Underlay configuration
set interfaces ethernet eth0 address 192.0.2.10/24

# VXLAN tunnel
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-address 192.0.2.10
set interfaces vxlan vxlan100 remote 192.0.2.20
set interfaces vxlan vxlan100 mtu 1450

# Overlay address
set interfaces vxlan vxlan100 address 10.100.0.1/24
set interfaces vxlan vxlan100 description 'VXLAN to Node2'

commit
```

**Node 2** (192.0.2.20):
```
# Underlay configuration
set interfaces ethernet eth0 address 192.0.2.20/24

# VXLAN tunnel
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-address 192.0.2.20
set interfaces vxlan vxlan100 remote 192.0.2.10
set interfaces vxlan vxlan100 mtu 1450

# Overlay address
set interfaces vxlan vxlan100 address 10.100.0.2/24
set interfaces vxlan vxlan100 description 'VXLAN to Node1'

commit
```

Now the nodes can communicate over the 10.100.0.0/24 overlay network.

### VXLAN Bridge for L2 Stretching (VK Cloud)

Stretching an L2 domain between two data centers:

**Site A** (198.51.100.10):
```
# VXLAN interface
set interfaces vxlan vxlan200 vni 200
set interfaces vxlan vxlan200 source-address 198.51.100.10
set interfaces vxlan vxlan200 remote 203.0.113.10
set interfaces vxlan vxlan200 mtu 1450

# Bridge with a local interface
set interfaces bridge br200
set interfaces bridge br200 member interface vxlan200
set interfaces bridge br200 member interface eth1
set interfaces bridge br200 description 'Stretched L2 VLAN 200'

# IP for inter-site routing (optional)
set interfaces bridge br200 address 10.200.0.1/24

commit
```

**Site B** (203.0.113.10):
```
# VXLAN interface
set interfaces vxlan vxlan200 vni 200
set interfaces vxlan vxlan200 source-address 203.0.113.10
set interfaces vxlan vxlan200 remote 198.51.100.10
set interfaces vxlan vxlan200 mtu 1450

# Bridge with a local interface
set interfaces bridge br200
set interfaces bridge br200 member interface vxlan200
set interfaces bridge br200 member interface eth1
set interfaces bridge br200 description 'Stretched L2 VLAN 200'

# IP for inter-site routing (optional)
set interfaces bridge br200 address 10.200.0.2/24

commit
```

Devices on eth1 at both sites are in the same L2 domain.

### Multi-tenant VXLAN (cloud provider)

Isolating traffic from different customers using separate VNIs:

```
# Customer A - VNI 1000
set interfaces vxlan vxlan1000 vni 1000
set interfaces vxlan vxlan1000 source-interface eth0
set interfaces vxlan vxlan1000 remote 192.0.2.20

set interfaces bridge brA
set interfaces bridge brA member interface vxlan1000
set interfaces bridge brA member interface eth1.100
set interfaces bridge brA address 10.10.0.1/24
set interfaces bridge brA description 'Tenant A'

# Customer B - VNI 2000
set interfaces vxlan vxlan2000 vni 2000
set interfaces vxlan vxlan2000 source-interface eth0
set interfaces vxlan vxlan2000 remote 192.0.2.20

set interfaces bridge brB
set interfaces bridge brB member interface vxlan2000
set interfaces bridge brB member interface eth1.200
set interfaces bridge brB address 10.20.0.1/24
set interfaces bridge brB description 'Tenant B'

# Customer C - VNI 3000
set interfaces vxlan vxlan3000 vni 3000
set interfaces vxlan vxlan3000 source-interface eth0
set interfaces vxlan vxlan3000 remote 192.0.2.20

set interfaces bridge brC
set interfaces bridge brC member interface vxlan3000
set interfaces bridge brC member interface eth1.300
set interfaces bridge brC address 10.30.0.1/24
set interfaces bridge brC description 'Tenant C'

commit
```

Each customer is fully isolated at the L2 level through a separate VNI.

### VXLAN Multicast (on-premises)

For on-premises data centers with multicast support:

```
# VXLAN with multicast
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-interface eth0
set interfaces vxlan vxlan100 group 239.1.1.100
set interfaces vxlan vxlan100 mtu 1450

# Bridge for local L2
set interfaces bridge br100
set interfaces bridge br100 member interface vxlan100
set interfaces bridge br100 member interface eth1
set interfaces bridge br100 member interface eth2

commit
```

Automatic VTEP discovery through the multicast group 239.1.1.100.

### VXLAN over loopback (recommended for production)

Using a loopback address for stability:

```
# Loopback interface
set interfaces loopback lo address 198.51.100.1/32

# VXLAN with source-address on the loopback
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-address 198.51.100.1
set interfaces vxlan vxlan100 remote 203.0.113.1
set interfaces vxlan vxlan100 mtu 1450

# Route to the remote loopback
set protocols static route 203.0.113.1/32 next-hop 192.0.2.1

commit
```

A loopback address does not depend on the state of physical interfaces.

### VXLAN with IPv6 underlay

VXLAN supports IPv6 for the underlay network:

```
# IPv6 underlay
set interfaces ethernet eth0 address 2001:db8:1::10/64

# VXLAN over IPv6
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-address 2001:db8:1::10
set interfaces vxlan vxlan100 remote 2001:db8:2::10
set interfaces vxlan vxlan100 mtu 1450

# IPv4 overlay
set interfaces vxlan vxlan100 address 10.100.0.1/24

commit
```

### VXLAN with firewall

Protecting VXLAN traffic with a firewall:

```
# VXLAN interface
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-interface eth0
set interfaces vxlan vxlan100 remote 192.0.2.20
set interfaces vxlan vxlan100 address 10.100.0.1/24

# Firewall for overlay traffic
set firewall ipv4 input filter rule 100 action accept
set firewall ipv4 input filter rule 100 source address 10.100.0.0/24
set firewall ipv4 input filter rule 100 description 'Allow VXLAN overlay'

# Firewall for underlay VXLAN (UDP 8472)
set firewall ipv4 input filter rule 200 action accept
set firewall ipv4 input filter rule 200 protocol udp
set firewall ipv4 input filter rule 200 destination port 8472
set firewall ipv4 input filter rule 200 source address 192.0.2.0/24
set firewall ipv4 input filter rule 200 description 'Allow VXLAN encapsulation'

commit
```

### VXLAN Full-Mesh (3 nodes)

Building a fully meshed topology for 3 nodes:

**Option 1: Separate VXLAN per pair** (not recommended):
Too many interfaces for a large number of nodes.

**Option 2: Multicast** (if supported):
```
# Identical configuration on all nodes
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-interface eth0
set interfaces vxlan vxlan100 group 239.0.0.100
set interfaces vxlan vxlan100 mtu 1450

set interfaces bridge br100
set interfaces bridge br100 member interface vxlan100
set interfaces bridge br100 address 10.100.0.X/24

commit
```

**Option 3: BGP EVPN** (best practice for production):
Use BGP EVPN for automatic distribution of MAC/IP routes and to build a VXLAN fabric.

## Integration with Cloud Providers

### Yandex Cloud specifics

**Limitations**:
- Multicast is not supported
- Use unicast VXLAN only
- MTU: 1450 is recommended (Yandex Cloud MTU = 1500)
- Security groups: allow UDP 8472 between nodes

**Security Group configuration**:
```
# In the Yandex Cloud console, create a rule:
Protocol: UDP
Port: 8472
Source: CIDR block with the VTEP node addresses
```

**Example for an HA cluster**:
```
# Node 1 (primary)
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-interface eth0
set interfaces vxlan vxlan100 remote 10.128.0.20
set interfaces vxlan vxlan100 address 172.16.0.1/24

# VRRP for HA
set high-availability vrrp group vxlan100 vrid 100
set high-availability vrrp group vxlan100 interface vxlan100
set high-availability vrrp group vxlan100 address 172.16.0.254/24
set high-availability vrrp group vxlan100 priority 200

commit
```

### VK Cloud specifics

**Limitations**:
- Multicast is not supported
- MTU: 1450 is recommended
- Firewall: allow UDP 8472

**Firewall configuration**:
In the VK Cloud panel:
```
Inbound traffic rule:
- Protocol: UDP
- Port: 8472
- Source: VTEP IP addresses
```

**Overlay for Kubernetes**:
```
# VXLAN for the Kubernetes CNI
set interfaces vxlan vxlan10 vni 10
set interfaces vxlan vxlan10 source-interface eth0
set interfaces vxlan vxlan10 remote 10.0.0.20

set interfaces bridge cni0
set interfaces bridge cni0 member interface vxlan10
set interfaces bridge cni0 address 10.244.0.1/24

# IP forwarding for pod routing
# VyOS enables forwarding automatically

commit
```

### General recommendations for clouds

1. **Use unicast VXLAN** - multicast is usually not supported
2. **MTU 1450** - to prevent fragmentation
3. **Security groups/Firewall** - allow UDP 8472 between VTEPs
4. **Source-address on a loopback** - for stability during failover
5. **Monitor the underlay** - check the reachability of remote VTEPs
6. **BGP EVPN** - for automation in large deployments

## UDP Port Configuration

Changing the default port (8472) to the IANA one (4789):

```
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 port 4789
set interfaces vxlan vxlan100 source-interface eth0
set interfaces vxlan vxlan100 remote 192.0.2.20
commit
```

**When to change the port**:
- Integration with hardware from other vendors (Cisco, Arista use 4789)
- Corporate standards require the IANA port
- Firewall rules are already configured for 4789

**Important**: The port must match on all VTEPs within the same VNI.

## MAC Address

Setting a static MAC address for the VXLAN interface:

```
set interfaces vxlan vxlan100 mac 00:50:56:00:00:01
commit
```

May be required for:
- Integration with legacy systems
- MAC-based licensing
- Debugging

## ARP and ND

### ARP Cache Timeout

```
set interfaces vxlan vxlan100 ip arp-cache-timeout 3600
commit
```

### Disable ARP

```
set interfaces vxlan vxlan100 ip disable-arp-filter
commit
```

### IPv6 Neighbor Discovery

```
set interfaces vxlan vxlan100 ipv6 dup-addr-detect-transmits 1
commit
```

## Operational Commands

### Viewing VXLAN Interfaces

```
show interfaces vxlan
```

Example output:
```
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
vxlan100         10.100.0.1/24                     u/u  VXLAN Overlay 100
vxlan200         10.200.0.1/24                     u/u  VXLAN Overlay 200
```

### Detailed VXLAN Information

```
show interfaces vxlan vxlan100
```

Example output:
```
vxlan100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP
    link/ether 00:50:56:00:12:34 brd ff:ff:ff:ff:ff:ff
    inet 10.100.0.1/24 brd 10.100.0.255 scope global vxlan100
    vxlan id 100 remote 192.0.2.20 dev eth0 srcport 0 0 dstport 8472
```

### VXLAN Statistics

```
show interfaces vxlan vxlan100 statistics
```

Example output:
```
RX:  bytes    packets  errors  dropped  overrun  mcast
     1048576  1024     0       0        0        0
TX:  bytes    packets  errors  dropped  carrier  collsns
     2097152  2048     0       0        0        0
```

### FDB (Forwarding Database)

Viewing the MAC addresses learned in VXLAN:

```
show bridge vxlan vxlan100 fdb
```

Example output:
```
port    mac addr           flags
vxlan100  00:0c:29:12:34:56  dst 192.0.2.20
vxlan100  00:0c:29:ab:cd:ef  dst 192.0.2.30
```

### Checking Connectivity

Ping over the overlay network:

```
ping 10.100.0.2 source-address 10.100.0.1
```

Traceroute:

```
traceroute 10.100.0.2 source-address 10.100.0.1
```

### Monitoring Encapsulated Traffic

```
monitor interfaces vxlan vxlan100 traffic
```

Capturing VXLAN packets:

```
monitor traffic interface eth0 filter 'udp port 8472'
```

## Troubleshooting

### VXLAN Interface Does Not Come Up

Check the source-interface:
```
show interfaces ethernet eth0
```

Make sure the source interface is in the UP state.

Check the configuration:
```
show configuration interfaces vxlan vxlan100
```

### No Connectivity over VXLAN

**1. Check underlay connectivity**:
```
ping 192.0.2.20 source-address 192.0.2.10
```

If the underlay does not work, VXLAN will not work either.

**2. Check the firewall/security groups**:
```
# On the remote node
sudo tcpdump -i eth0 udp port 8472
```

If packets do not arrive, the problem is in the firewall.

**3. Check the VNI**:
The VNI must match on both ends of the tunnel.

**4. Check the MTU**:
```
ping 10.100.0.2 size 1400 do-not-fragment
```

If packets do not pass, reduce the MTU.

### FDB Is Not Learning (MACs Not Visible)

Check the bridge configuration:
```
show bridge brX fdb
```

Make sure the VXLAN interface is in the bridge:
```
show bridge brX
```

Check the aging time:
```
show configuration interfaces bridge brX
```

### Broadcast Storm in VXLAN

Possible causes:
- A loop in the overlay topology
- Incorrect STP configuration
- Duplicate MAC addresses

**Solution**:
```
# Enable STP on the bridge
set interfaces bridge br100 stp
commit

# Check STP
show bridge br100 spanning-tree
```

### High Packet Fragmentation

Reduce the VXLAN MTU:

```
set interfaces vxlan vxlan100 mtu 1400
commit
```

Or increase the underlay MTU (if possible):

```
set interfaces ethernet eth0 mtu 1600
commit
```

### Remote VTEP Unreachable

Check the route:
```
show ip route 192.0.2.20
```

Check ARP:
```
show arp interface eth0
```

Traceroute to the remote VTEP:
```
traceroute 192.0.2.20
```

### VXLAN Works Only in One Direction

Check for a symmetric configuration on both ends:
- The VNI must match
- The UDP port must match
- The remote IP must point to the correct peer

### Poor VXLAN Performance

**1. Check the CPU load**:
```
show system cpu
```

VXLAN encapsulation consumes CPU.

**2. Enable offloading** (if supported):
```
show interfaces ethernet eth0 offload
```

**3. Use jumbo frames**:
```
# Underlay
set interfaces ethernet eth0 mtu 9000

# VXLAN
set interfaces vxlan vxlan100 mtu 8950

commit
```

**4. Check the underlay bandwidth**:
There may be a bottleneck in the physical network.

## MTU Calculation

### Standard MTU (1500)

```
VXLAN MTU = Underlay MTU - VXLAN Overhead
VXLAN MTU = 1500 - 50 = 1450
```

**Overhead components**:
- Outer Ethernet: 14 bytes
- Outer IP: 20 bytes (IPv4) or 40 bytes (IPv6)
- UDP: 8 bytes
- VXLAN: 8 bytes
- **Total**: 50 bytes (IPv4), 70 bytes (IPv6)

### Jumbo Frames (9000)

```
VXLAN MTU = 9000 - 50 = 8950
```

### With a VLAN Tag

```
VXLAN MTU = 1500 - 50 - 4 = 1446
```

A VLAN tag adds 4 bytes.

### Automatic MTU Detection

```
# Path MTU Discovery is enabled by default
show interfaces vxlan vxlan100
```

## Performance

### Optimization Recommendations

**1. Hardware offloading**:
Use a NIC with VXLAN offload support.

**2. Jumbo frames**:
Increase the MTU to 9000 on the underlay and 8950 on the overlay.

**3. Source on a loopback**:
Use loopback addresses for VTEPs - more stable during failover.

**4. Multipath routing**:
Use ECMP to load-balance VXLAN traffic.

**5. CPU pinning**:
For critical workloads, configure CPU affinity.

### Performance Testing

**Throughput**:
```
# On the server
iperf3 -s -B 10.100.0.1

# On the client
iperf3 -c 10.100.0.1 -t 60 -P 4
```

**Latency**:
```
ping 10.100.0.2 -i 0.2 -c 1000
```

**Packet loss**:
```
mtr 10.100.0.2 -c 100
```

## Security

### VXLAN Encryption

By default, VXLAN **does not encrypt** traffic. To encrypt it:

**Option 1: IPsec over VXLAN**:
```
# Create an IPsec tunnel between the VTEPs
set vpn ipsec site-to-site peer 192.0.2.20 ...

# VXLAN over the IPsec tunnel
set interfaces vxlan vxlan100 source-interface ipsec0
```

**Option 2: WireGuard underlay**:
```
# WireGuard tunnel
set interfaces wireguard wg0 ...

# VXLAN over WireGuard
set interfaces vxlan vxlan100 source-interface wg0
```

### Firewall for VXLAN

**Protecting the underlay**:
```
# Allow VXLAN only from known VTEPs
set firewall ipv4 input filter rule 100 action accept
set firewall ipv4 input filter rule 100 protocol udp
set firewall ipv4 input filter rule 100 destination port 8472
set firewall ipv4 input filter rule 100 source address 192.0.2.0/24

set firewall ipv4 input filter rule 110 action drop
set firewall ipv4 input filter rule 110 protocol udp
set firewall ipv4 input filter rule 110 destination port 8472

commit
```

**Protecting the overlay**:
```
# Firewall for overlay traffic
set firewall ipv4 forward filter rule 200 action accept
set firewall ipv4 forward filter rule 200 inbound-interface name vxlan100
set firewall ipv4 forward filter rule 200 state established
set firewall ipv4 forward filter rule 200 state related

commit
```

### Rate Limiting

Protection against flooding:

```
set firewall ipv4 input filter rule 300 action accept
set firewall ipv4 input filter rule 300 protocol udp
set firewall ipv4 input filter rule 300 destination port 8472
set firewall ipv4 input filter rule 300 limit rate 1000/second
commit
```

## Monitoring

### Key Metrics

**1. Interface statistics**:
```
show interfaces vxlan vxlan100 statistics
```

Track:
- RX/TX errors
- Dropped packets
- Overruns

**2. FDB size**:
```
show bridge brX fdb | count
```

**3. CPU usage**:
```
show system cpu
```

VXLAN encapsulation consumes CPU.

**4. Underlay health**:
```
ping 192.0.2.20 source-address 192.0.2.10
```

### Logging

Enabling debug logs for VXLAN:

```
# In VyOS 1.4
set system syslog global facility local7 level debug

# In VyOS 1.5
# See the logging documentation
```

Viewing the logs:
```
show log | match vxlan
```

## Integration

### With BGP EVPN

BGP EVPN automates the distribution of MAC/IP routes across a VXLAN fabric:

```
# Loopback for the VTEP
set interfaces loopback lo address 192.0.2.1/32

# VXLAN
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-address 192.0.2.1

# Bridge
set interfaces bridge br100
set interfaces bridge br100 member interface vxlan100

# BGP for the underlay
set protocols bgp system-as 65000
set protocols bgp neighbor 10.0.0.1 remote-as 65000

# BGP EVPN
set protocols bgp address-family l2vpn-evpn advertise-all-vni
set protocols bgp address-family l2vpn-evpn advertise-default-gw
set protocols bgp address-family l2vpn-evpn neighbor 10.0.0.1 activate

commit
```

### With OSPF (underlay)

OSPF for routing between VTEPs:

```
# Loopback
set interfaces loopback lo address 192.0.2.1/32

# OSPF
set protocols ospf area 0 network 192.0.2.1/32
set protocols ospf area 0 network 10.0.0.0/24

# VXLAN with source on the loopback
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 source-address 192.0.2.1
set interfaces vxlan vxlan100 remote 192.0.2.2

commit
```

OSPF advertises the loopback addresses, providing underlay connectivity.

### With VRF

Isolating the VXLAN overlay in a separate VRF:

```
# VRF for the tenant
set vrf name TENANT table 100

# VXLAN in the VRF
set interfaces vxlan vxlan100 vni 100
set interfaces vxlan vxlan100 vrf TENANT
set interfaces vxlan vxlan100 address 10.100.0.1/24

commit
```

### With QoS

Prioritizing VXLAN traffic:

```
# QoS for the VXLAN underlay
set qos policy shaper VXLAN-QOS bandwidth 1gbit
set qos policy shaper VXLAN-QOS class 10 match vxlan ip protocol udp
set qos policy shaper VXLAN-QOS class 10 match vxlan ip destination port 8472
set qos policy shaper VXLAN-QOS class 10 bandwidth 800mbit

set interfaces ethernet eth0 qos egress VXLAN-QOS

commit
```

## Technology Comparison

### VXLAN vs VLAN

| Characteristic | VXLAN | VLAN |
|---------------|-------|------|
| Identifier | 24 bits (16M) | 12 bits (4096) |
| Scalability | Very high | Limited |
| Works across L3 | Yes | No |
| Overhead | 50 bytes | 4 bytes |
| Complexity | High | Low |
| Use case | Data center, cloud | Campus, enterprise |

### VXLAN vs GRE

| Characteristic | VXLAN | GRE |
|---------------|-------|-----|
| Layer | L2 over L3 | L3 over L3 |
| Multicast | Supported | No |
| NAT traversal | Yes (UDP) | Limited |
| Identifier | VNI (24 bits) | Key (32 bits) |
| Scalability | High | Medium |

### VXLAN vs MPLS L2VPN

| Characteristic | VXLAN | MPLS L2VPN |
|---------------|-------|-----------|
| Protocol | UDP | MPLS |
| Complexity | Medium | High |
| Cost | Low | High |
| Control plane | BGP EVPN/Multicast | LDP/BGP |
| Use case | Cloud/DC | Service provider |

## Best Practices

1. **MTU planning** - always subtract 50 bytes from the underlay MTU
2. **Source on a loopback** - for stability during failover
3. **Unicast for clouds** - multicast is usually not supported
4. **Underlay firewall** - allow UDP 8472 only from known VTEPs
5. **FDB monitoring** - track the size of the forwarding database
6. **BGP EVPN for production** - automation in large deployments
7. **Jumbo frames** - if the underlay supports them
8. **VNI plan** - document VNI assignments
9. **Overlay security** - use a firewall on overlay interfaces
10. **Testing** - test failover scenarios

## Limitations

- VXLAN adds 50 bytes of overhead (IPv4) or 70 bytes (IPv6)
- Encapsulation consumes CPU (hardware offload is recommended)
- Multicast VXLAN does not work in most cloud providers
- Careful MTU planning is required
- FDB tables can grow large in big networks
- Debugging is harder than with ordinary L2/L3 networks

## Examples for Specific Scenarios

### Kubernetes Overlay

```
# VXLAN for the Kubernetes pod network
set interfaces vxlan vxlan-pods vni 1000
set interfaces vxlan vxlan-pods source-interface eth0
set interfaces vxlan vxlan-pods remote 10.0.0.20
set interfaces vxlan vxlan-pods mtu 1450

set interfaces bridge kube-bridge
set interfaces bridge kube-bridge member interface vxlan-pods
set interfaces bridge kube-bridge address 10.244.0.1/24

commit
```

### Multi-DC DCI (Data Center Interconnect)

```
# DC1 VTEP
set interfaces loopback lo address 198.51.100.1/32
set interfaces vxlan vxlan-dci vni 5000
set interfaces vxlan vxlan-dci source-address 198.51.100.1
set interfaces vxlan vxlan-dci remote 203.0.113.1
set interfaces vxlan vxlan-dci mtu 1450

# QoS for DCI
set qos policy shaper DCI bandwidth 10gbit

commit
```

### NFV Service Chaining

```
# VNF1 -> VXLAN -> VNF2 -> VXLAN -> VNF3

# VXLAN segments
set interfaces vxlan vxlan-vnf1-vnf2 vni 2001
set interfaces vxlan vxlan-vnf1-vnf2 source-interface eth0
set interfaces vxlan vxlan-vnf1-vnf2 remote 192.0.2.20

set interfaces vxlan vxlan-vnf2-vnf3 vni 2002
set interfaces vxlan vxlan-vnf2-vnf3 source-interface eth0
set interfaces vxlan vxlan-vnf2-vnf3 remote 192.0.2.30

# Bridges for service chaining
set interfaces bridge br-vnf1
set interfaces bridge br-vnf1 member interface vxlan-vnf1-vnf2
set interfaces bridge br-vnf1 member interface eth1

set interfaces bridge br-vnf2
set interfaces bridge br-vnf2 member interface vxlan-vnf2-vnf3
set interfaces bridge br-vnf2 member interface eth2

commit
```

## Next Steps

- [Bridge interfaces](/docs/vyos/interfaces/vyos-bridge/) - L2 bridging and VLAN-aware mode
- [GRE tunnels](/docs/vyos/interfaces/vyos-tunnel/) - an alternative tunneling technology
- [VRF](/docs/vyos/vrf/) - routing table isolation for multi-tenancy
- [BGP](/docs/vyos/routing/vyos-bgp/) - the control plane for a VXLAN fabric
- [Firewall](/docs/vyos/firewall/) - protecting the VXLAN overlay and underlay
- [QoS](/docs/vyos/qos/) - prioritizing VXLAN traffic

## References

- [RFC 7348 - VXLAN](https://datatracker.ietf.org/doc/html/rfc7348)
- [VXLAN Deployment Guide](https://docs.vyos.io/en/latest/configuration/interfaces/vxlan.html)
- [BGP EVPN for VXLAN](https://datatracker.ietf.org/doc/html/rfc8365)
- [Yandex Cloud Networking](https://cloud.yandex.ru/docs/vpc/)
- [VK Cloud Networking](https://cloud.vk.com/docs/networks)