Documentation
VyOS Documentation
VyOS Administrator Guide
VPP (Vector Packet Processing) in VyOS

VPP (Vector Packet Processing) in VyOS

Vector Packet Processing (VPP) is a high-performance dataplane framework from the FD.io project that can be used in VyOS instead of the standard Linux kernel networking stack to achieve significantly higher packet-processing performance.

Introduction to VPP

What is VPP?

VPP (Vector Packet Processing) is a modular, extensible framework for processing packets in userspace, developed by the FD.io (Fast Data Project) under the Linux Foundation.

Key features:

  • Vector processing: Processes packets in groups (vectors) rather than one at a time
  • Userspace: Runs in userspace, bypassing the kernel networking stack
  • Zero-copy: Minimizes data copying between user and kernel space
  • DPDK integration: Uses Intel DPDK for direct access to network cards
  • Graph architecture: Modular architecture based on a packet-processing graph
  • High performance: 10-100x better performance compared to kernel networking

Advantages of VPP

Performance:

  • Throughput: up to 200+ Gbps on commodity hardware
  • Latency: microsecond packet-processing latency
  • PPS: millions of packets per second

Efficiency:

  • Optimized CPU cache usage
  • Minimized context switches
  • Efficient use of CPU cores

Flexibility:

  • Modular architecture with plugins
  • Programmability through plugins
  • Support for many protocols

When to Use VPP?

Recommended for:

  • High-throughput scenarios (10+ Gbps)
  • NFV (Network Function Virtualization)
  • Service provider edge
  • High-performance routing/switching
  • vRouter in cloud environments
  • IPsec VPN under heavy load

Not recommended for:

  • SOHO/small offices (overkill)
  • Systems with limited resources
  • Simple configurations
  • Cases where kernel networking is sufficient

VPP Architecture in VyOS

Standard Linux Networking vs VPP

Linux Kernel Networking:

Application
    ↓
 Socket API
    ↓
Kernel Network Stack
    ↓
Network Driver
    ↓
 NIC Hardware

VPP Dataplane:

Application (via VPP API)
    ↓
VPP (userspace)
    ↓
DPDK PMD (Poll Mode Driver)
    ↓
NIC Hardware (direct access)

VPP Integration with VyOS

VyOS can use VPP as an alternative dataplane:

VyOS CLI/API
    ↓
VyOS Configuration Daemon
    ↓
VPP Control Plane Integration
    ↓
VPP Dataplane (packet processing)

Components:

  • VPP daemon: The core packet-processing process
  • VPP plugins: Functionality extensions (NAT, ACL, IPsec, etc.)
  • Control plane integration: Integration with FRR for routing protocols
  • Configuration management: VyOS CLI → VPP API

Installing and Configuring VPP

System Requirements

Hardware:

  • CPU: x86_64 with SSE4.2 and AVX2 support (recommended)
  • RAM: 4GB minimum (8GB+ recommended)
  • NIC: DPDK-compatible network cards (Intel, Mellanox, etc.)
  • Hugepages: enabled in BIOS/kernel

Software:

  • VyOS 1.4+ (rolling release recommended)
  • Linux kernel with hugepages support
  • DPDK-compatible drivers

Checking NIC Compatibility

# Check the PCI IDs of the network cards
lspci | grep Ethernet

# Check DPDK compatibility
# Intel 82599, X710, XL710 - excellent support
# Intel e1000e (igb) - basic support
# Mellanox ConnectX-4/5 - excellent support

# Check the current driver
ethtool -i eth0 | grep driver

Configuring Hugepages

VPP requires hugepages for efficient memory management:

# Check current hugepages
cat /proc/meminfo | grep Huge

# Configure hugepages (2MB pages, 1024 pages = 2GB)
configure
set system option hugepages hugepage-size 2048
set system option hugepages count 1024
commit
save

# Alternatively - via kernel parameters
sudo vi /boot/grub/grub.cfg
# Add to the kernel line:
# hugepagesz=2M hugepages=1024 default_hugepagesz=2M

# Reboot to apply
reboot

Installing VPP (if not preinstalled)

# VPP may be preinstalled in some VyOS images
# Check whether it is present
dpkg -l | grep vpp

# If not installed - install from the repository
sudo apt-get update
sudo apt-get install vpp vpp-plugin-core vpp-plugin-dpdk

# Check the version
vppctl show version

Basic VPP Configuration

Enabling the VPP Dataplane

configure

# Enable VPP
set system dataplane vpp

# Specify the interfaces for VPP
set system dataplane vpp interface eth0
set system dataplane vpp interface eth1

# Configure CPU cores for VPP
set system dataplane vpp cpu main-core 0
set system dataplane vpp cpu corelist-workers 1-3

commit
save

# Reboot to apply
reboot

Checking VPP Status

# After reboot - check VPP
vppctl show version
vppctl show hardware
vppctl show interface

# Verify that the interfaces are picked up by VPP
vppctl show hardware-interfaces

Working with VPP Interfaces

Managing Interfaces through VPP

Once VPP is enabled, interfaces are managed through the VPP control plane:

# The VyOS CLI continues to work for configuration
configure
set interfaces ethernet eth0 address 192.168.1.1/24
commit

# Internally, VyOS translates the commands into VPP API calls
# Check via vppctl
vppctl show interface address

Checking Interface Performance

# VPP interface statistics
vppctl show interface

# Detailed statistics
vppctl show hardware-interfaces

# Errors and drops
vppctl show errors

# Node graph statistics
vppctl show runtime

VPP for Specific Functions

IPsec with VPP

VPP provides hardware-accelerated IPsec:

configure

# Enable the VPP IPsec plugin
set system dataplane vpp plugin dpdk enable
set system dataplane vpp plugin crypto enable

commit
save

# Configure IPsec as usual through the VyOS CLI
set vpn ipsec ike-group IKE-VPP proposal 1 encryption aes256
set vpn ipsec ike-group IKE-VPP proposal 1 hash sha256
set vpn ipsec ike-group IKE-VPP proposal 1 dh-group 14

set vpn ipsec esp-group ESP-VPP proposal 1 encryption aes256
set vpn ipsec esp-group ESP-VPP proposal 1 hash sha256

set vpn ipsec site-to-site peer 203.0.113.1 authentication mode pre-shared-secret
set vpn ipsec site-to-site peer 203.0.113.1 authentication pre-shared-secret 'secret'
set vpn ipsec site-to-site peer 203.0.113.1 ike-group IKE-VPP
set vpn ipsec site-to-site peer 203.0.113.1 local-address 203.0.113.2
set vpn ipsec site-to-site peer 203.0.113.1 tunnel 1 esp-group ESP-VPP
set vpn ipsec site-to-site peer 203.0.113.1 tunnel 1 local prefix 192.168.1.0/24
set vpn ipsec site-to-site peer 203.0.113.1 tunnel 1 remote prefix 192.168.2.0/24

commit
save

Checking VPP IPsec:

# Check IPsec via VPP
vppctl show ipsec all

# IPsec statistics
vppctl show ipsec sa
vppctl show ipsec tunnel

IPsec performance with VPP:

  • Up to 10-50 Gbps IPsec throughput (depends on hardware)
  • AES-NI hardware acceleration
  • Parallel processing of multiple tunnels

NAT with VPP

The VPP NAT plugin provides high-performance NAT:

configure

# Enable the VPP NAT plugin
set system dataplane vpp plugin nat enable

# Configure NAT as usual
set nat source rule 100 outbound-interface eth0
set nat source rule 100 source address 192.168.1.0/24
set nat source rule 100 translation address masquerade

commit
save

Checking VPP NAT:

# NAT sessions in VPP
vppctl show nat44 sessions

# NAT statistics
vppctl show nat44 statistics

# NAT addresses
vppctl show nat44 addresses

NAT performance with VPP:

  • Millions of NAT sessions
  • High PPS for new connections
  • Scales with the number of CPU cores

ACL/Firewall with VPP

The VPP ACL plugin for firewall functions:

configure

# Enable the VPP ACL plugin
set system dataplane vpp plugin acl enable

# Configure the firewall through the VyOS CLI as usual
set firewall name WAN_LOCAL default-action drop
set firewall name WAN_LOCAL rule 10 action accept
set firewall name WAN_LOCAL rule 10 state established enable
set firewall name WAN_LOCAL rule 10 state related enable

set interfaces ethernet eth0 firewall local name WAN_LOCAL

commit
save

Checking VPP ACL:

# ACL rules in VPP
vppctl show acl-plugin acl

# ACL statistics
vppctl show acl-plugin sessions

Monitoring and Debugging VPP

Basic Monitoring

# General information
vppctl show version
vppctl show hardware
vppctl show memory

# CPU usage
vppctl show runtime

# Interfaces
vppctl show interface
vppctl show interface address

# Errors
vppctl show errors

# Packet graph
vppctl show vlib graph

Detailed Statistics

# Node runtime statistics
vppctl show runtime verbose

# Per-node statistics
vppctl show node counters

# Buffer usage
vppctl show buffers

# Thread statistics
vppctl show threads

Packet Tracing

# Enable packet trace (first 100 packets)
vppctl trace add dpdk-input 100

# Show the trace
vppctl show trace

# Clear the trace
vppctl clear trace

Performance Tuning

# Optimize CPU cores
vppctl set interface rx-mode eth0 polling

# Configure buffer size
# In /etc/vpp/startup.conf
unix {
  cli-listen /run/vpp/cli.sock
}
cpu {
  main-core 0
  corelist-workers 1-7
}
dpdk {
  dev 0000:03:00.0
  dev 0000:04:00.0
  num-mbufs 128000
  socket-mem 2048,2048
}

Troubleshooting VPP

Problem: VPP Does Not Start

Diagnostics:

# Check the status of the VPP service
sudo systemctl status vpp

# VPP logs
sudo journalctl -u vpp -n 100

# VPP startup log
sudo cat /var/log/vpp/vpp.log

Common causes:

  1. Hugepages not configured
# Check
cat /proc/meminfo | grep Huge

# If HugePages_Total = 0, configure hugepages
echo 1024 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
  1. DPDK cannot access the NIC
# Check the PCI binding
sudo dpdk-devbind.py --status

# Bind the interface to the DPDK driver
sudo dpdk-devbind.py --bind=vfio-pci 0000:03:00.0
  1. Insufficient memory
# Check available memory
free -h

# Increase socket-mem in /etc/vpp/startup.conf

Problem: Low Performance

Diagnostics:

# Check CPU usage
vppctl show runtime

# Check packet drops
vppctl show hardware-interfaces

# Check errors
vppctl show errors

Solutions:

  1. Increase worker cores
# In /etc/vpp/startup.conf
cpu {
  main-core 0
  corelist-workers 1-7  # More cores
}
  1. Optimize RX mode
vppctl set interface rx-mode eth0 polling
vppctl set interface rx-mode eth1 polling
  1. Increase buffers
# In /etc/vpp/startup.conf
dpdk {
  num-mbufs 256000  # Increase
}

Problem: Interfaces Are Not Available

Diagnostics:

# Check the hardware interfaces
vppctl show hardware-interfaces

# Check the PCI devices
lspci | grep Ethernet

# Check the DPDK binding
sudo dpdk-devbind.py --status

Solution:

# Bind the interfaces to DPDK
sudo dpdk-devbind.py --bind=vfio-pci 0000:03:00.0
sudo dpdk-devbind.py --bind=vfio-pci 0000:04:00.0

# Restart VPP
sudo systemctl restart vpp

Practical VPP Scenarios

Scenario 1: High-throughput Router with VPP

Goal: A router with 40+ Gbps throughput for the ISP edge

Configuration:

configure

# Enable VPP
set system dataplane vpp

# Configure the interfaces
set system dataplane vpp interface eth0
set system dataplane vpp interface eth1

# CPU allocation (8 cores)
set system dataplane vpp cpu main-core 0
set system dataplane vpp cpu corelist-workers 1-7

# Interfaces
set interfaces ethernet eth0 address 203.0.113.2/30
set interfaces ethernet eth0 description 'Upstream'

set interfaces ethernet eth1 address 192.168.1.1/24
set interfaces ethernet eth1 description 'Internal'

# Routing
set protocols static route 0.0.0.0/0 next-hop 203.0.113.1

# NAT with VPP
set system dataplane vpp plugin nat enable
set nat source rule 100 outbound-interface eth0
set nat source rule 100 source address 192.168.1.0/24
set nat source rule 100 translation address masquerade

commit
save
reboot

Checking performance:

# After reboot
# Run an iperf3 test
# Server on the internal network
iperf3 -s

# Client through the router
iperf3 -c <server-ip> -P 10 -t 60

# Monitor VPP
vppctl show runtime
vppctl show interface
vppctl show nat44 sessions

Scenario 2: IPsec VPN Gateway with VPP Acceleration

Goal: A VPN gateway with 10+ Gbps IPsec throughput

Configuration:

configure

# VPP with the crypto plugin
set system dataplane vpp
set system dataplane vpp plugin dpdk enable
set system dataplane vpp plugin crypto enable
set system dataplane vpp plugin nat enable

set system dataplane vpp interface eth0
set system dataplane vpp interface eth1

set system dataplane vpp cpu main-core 0
set system dataplane vpp cpu corelist-workers 1-15  # 16 cores total

# Interfaces
set interfaces ethernet eth0 address 203.0.113.10/24
set interfaces ethernet eth0 description 'WAN'

set interfaces ethernet eth1 address 192.168.1.1/24
set interfaces ethernet eth1 description 'LAN'

# Multiple IPsec tunnels
# Tunnel 1
set vpn ipsec site-to-site peer 203.0.113.20 authentication mode pre-shared-secret
set vpn ipsec site-to-site peer 203.0.113.20 authentication pre-shared-secret 'secret1'
set vpn ipsec site-to-site peer 203.0.113.20 ike-group IKE-VPP
set vpn ipsec site-to-site peer 203.0.113.20 local-address 203.0.113.10
set vpn ipsec site-to-site peer 203.0.113.20 tunnel 1 esp-group ESP-VPP
set vpn ipsec site-to-site peer 203.0.113.20 tunnel 1 local prefix 192.168.1.0/24
set vpn ipsec site-to-site peer 203.0.113.20 tunnel 1 remote prefix 192.168.2.0/24

# Tunnel 2
set vpn ipsec site-to-site peer 203.0.113.30 authentication mode pre-shared-secret
set vpn ipsec site-to-site peer 203.0.113.30 authentication pre-shared-secret 'secret2'
set vpn ipsec site-to-site peer 203.0.113.30 ike-group IKE-VPP
set vpn ipsec site-to-site peer 203.0.113.30 local-address 203.0.113.10
set vpn ipsec site-to-site peer 203.0.113.30 tunnel 1 esp-group ESP-VPP
set vpn ipsec site-to-site peer 203.0.113.30 tunnel 1 local prefix 192.168.1.0/24
set vpn ipsec site-to-site peer 203.0.113.30 tunnel 1 remote prefix 192.168.3.0/24

commit
save
reboot

Monitoring IPsec performance:

# IPsec tunnels
vppctl show ipsec all

# IPsec throughput
vppctl show interface

# Crypto engines
vppctl show crypto engines

Scenario 3: NFV vRouter in the Cloud

Goal: A virtual router in a cloud environment with VPP for high performance

Characteristics:

  • Runs inside a virtual machine
  • Virtio interfaces with DPDK
  • Optimized for a cloud environment

Configuration:

configure

# VPP with virtio support
set system dataplane vpp
set system dataplane vpp interface eth0
set system dataplane vpp interface eth1

# Fewer cores for a VM
set system dataplane vpp cpu main-core 0
set system dataplane vpp cpu corelist-workers 1-3

# Interfaces
set interfaces ethernet eth0 address dhcp
set interfaces ethernet eth0 description 'Cloud-WAN'

set interfaces ethernet eth1 address 10.0.1.1/24
set interfaces ethernet eth1 description 'Cloud-LAN'

# Cloud-specific routing
set protocols static route 0.0.0.0/0 next-hop dhcp-interface eth0

# NAT
set nat source rule 100 outbound-interface eth0
set nat source rule 100 source address 10.0.1.0/24
set nat source rule 100 translation address masquerade

commit
save
reboot

Performance Comparison: Linux Kernel vs VPP

Test Configuration

Hardware:

  • CPU: Intel Xeon Silver 4214 (12 cores, 24 threads)
  • RAM: 64GB DDR4
  • NIC: Intel X710 (10 Gbps)

Test setup:

  • Simple routing (no NAT, no firewall)
  • 64-byte packets (worst case)
  • 1500-byte packets (typical)

Test Results

Linux Kernel Networking:

64-byte packets:  ~2 Mpps (2 Gbps)
1500-byte packets: ~8 Gbps
Latency: 100-500 µs
CPU usage: 100% on 1 core at 8 Gbps

VPP Dataplane:

64-byte packets:  ~10 Mpps (10 Gbps)
1500-byte packets: ~40 Gbps (with 4 worker cores)
Latency: 10-50 µs
CPU usage: ~40% на 4 cores at 40 Gbps

Conclusion:

  • VPP: 5x better PPS (packets per second)
  • VPP: 5x better throughput
  • VPP: 10x lower latency
  • VPP: Scales better with cores

When Is the Gain Greatest?

VPP shows the biggest improvement in:

  1. High-PPS scenarios (small packets)
  2. IPsec encryption (hardware acceleration)
  3. A large number of NAT sessions
  4. Stateful firewall with many rules
  5. A combination of functions (routing + NAT + IPsec + ACL)

VPP Limitations in VyOS

Current Limitations

  1. Not all VyOS features are supported with VPP:

    • Some legacy protocols may not work
    • Specific kernel-based features are unavailable

  2. Hardware requirements:

    • DPDK-compatible NICs required for maximum performance
    • Hugepages are mandatory
    • Enough CPU cores for worker threads

  3. Complexity:

    • More complex setup and troubleshooting
    • Requires an understanding of the VPP architecture
    • Debugging is harder than with kernel networking

  4. Integration:

    • Some third-party tools may not work with VPP
    • Standard Linux tools (tcpdump, iptables) do not work directly with VPP

Alternative Solutions

If VPP is too complex, consider:

  1. Kernel optimizations:

    • XDP (eXpress Data Path)
    • TC (Traffic Control) offload
    • RSS (Receive Side Scaling) tuning

  2. Hardware offload:

    • SmartNICs (Mellanox, Netronome)
    • FPGA-based solutions

  3. Commercial solutions:

    • Cisco CSR 1000v with VPP
    • VyOS Professional with VPP support

Best Practices for VPP

Planning

  1. Assess the need for VPP:

    • Do you really need that throughput?
    • Is kernel networking sufficient?
    • Do you have suitable hardware?

  2. Plan the resources:

    • CPU cores: minimum 4 (main + 3 workers)
    • RAM: minimum 8GB for hugepages
    • NIC: DPDK-compatible

  3. Test in staging:

    • Always test the VPP configuration before production
    • Verify all required features
    • Measure the real-world performance

Configuration

  1. Allocate CPU cores correctly:
# Isolate cores for VPP
# In /etc/default/grub
GRUB_CMDLINE_LINUX="isolcpus=1-7"

# VPP uses the isolated cores
set system dataplane vpp cpu corelist-workers 1-7
  1. Optimize hugepages:
# Use 1GB hugepages for large deployments
echo 8 | sudo tee /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
  1. Monitoring:
# Regularly check VPP stats
vppctl show runtime
vppctl show errors
vppctl show hardware-interfaces

Operational Practices

  1. Configuration backup:

    • The VyOS configuration backup includes the VPP settings
    • Additionally back up /etc/vpp/startup.conf

  2. Performance monitoring:

    • Integration with Prometheus/Grafana
    • VPP metrics export
    • Alerting on packet drops

  3. Updates:

    • Test VPP updates in staging
    • Read the release notes for breaking changes
    • Plan maintenance windows

Conclusion

The VPP dataplane in VyOS provides a significant performance improvement for high-throughput scenarios:

Key advantages:

  • 5-10x throughput improvement
  • 10x latency reduction
  • Better scaling with CPU cores
  • Hardware-accelerated cryptography

When to use it:

  • ISP edge routers (10+ Gbps)
  • VPN gateways under heavy load
  • NFV/SDN deployments
  • Cloud vRouters
  • High-performance NAT gateways

When not to use it:

  • SOHO/small offices (overkill)
  • Limited resources
  • Simple scenarios
  • No suitable hardware

VPP is a powerful tool, but it requires careful planning, the right hardware, and an understanding of its architecture. For most scenarios, the standard Linux kernel networking in VyOS is sufficient and simpler to manage. VPP should be considered only when extreme performance is required and the corresponding resources are available to support it.

Additional Resources

Reviewed by OpenNix LLC · Last updated on
VyOS - Configuration Blueprints and Architecture