# Inter-VRF Routing over VRF Lite

> VRF Lite configuration with route leaking between VRFs for multi-tenant isolation in Yandex Cloud and VK Cloud environments

Source: https://opennix.org/en/docs/vyos/examples/inter-vrf-routing/


Inter-VRF routing enables controlled route exchange between isolated VRF (Virtual Routing and Forwarding) instances, providing segmentation with selective connectivity.

## Use Case

### Applicability

- **Multi-Tenant Cloud**: Customer isolation with controlled access to shared services
- **Enterprise Segmentation**: Separation of Production / Development / Testing environments
- **Service Provider**: Carrier VPN with partial route leaking
- **Security Zones**: DMZ, Internal, Management with selective access

### Benefits

1. **Network Isolation**: Full isolation of routing tables between tenants
2. **Selective Connectivity**: Controlled access to shared services (DNS, monitoring)
3. **IP Overlap**: Different VRFs can use identical IP ranges
4. **Flexible Policy**: Granular control through route-maps and prefix-lists
5. **Scalability**: Support for many VRFs on a single router

## Network Topology

### Basic topology (3 VRFs)

```
                    ┌─────────────────────────────────┐
                    │       VyOS Router               │
                    │                                 │
                    │  ┌────────────────────────┐     │
                    │  │   VRF PRODUCTION       │     │
                    │  │   RD: 65000:100        │     │
                    │  │   10.100.0.0/16        │     │
                    │  └──────────┬─────────────┘     │
                    │             │ Route Leak        │
                    │  ┌──────────┴─────────────┐     │
                    │  │   VRF SHARED           │     │
                    │  │   RD: 65000:999        │     │
                    │  │   10.255.0.0/24        │     │
                    │  │   (DNS, NTP, Monitor)  │     │
                    │  └──────────┬─────────────┘     │
                    │             │ Route Leak        │
                    │  ┌──────────┴─────────────┐     │
                    │  │   VRF DEVELOPMENT      │     │
                    │  │   RD: 65000:200        │     │
                    │  │   10.200.0.0/16        │     │
                    │  └────────────────────────┘     │
                    │                                 │
                    │  eth1 ─ PRODUCTION             │
                    │  eth2 ─ DEVELOPMENT            │
                    │  eth3 ─ SHARED                 │
                    └─────────────────────────────────┘
```

### Yandex Cloud Multi-Tenant

```
┌──────────────────────────────────────────┐
│      Yandex Cloud VPC                    │
│                                          │
│  ┌────────────────────────────────────┐  │
│  │       VyOS Gateway                 │  │
│  │                                    │  │
│  │  VRF CUSTOMER-A ◄──┐               │  │
│  │  (10.1.0.0/16)     │               │  │
│  │                    │               │  │
│  │  VRF CUSTOMER-B ◄──┼─► SHARED     │  │
│  │  (10.2.0.0/16)     │    SERVICES  │  │
│  │                    │    (DNS/NTP) │  │
│  │  VRF CUSTOMER-C ◄──┘               │  │
│  │  (10.3.0.0/16)                     │  │
│  └────────────────────────────────────┘  │
│                                          │
│  Customer A Subnet ◄─── eth1            │
│  Customer B Subnet ◄─── eth2            │
│  Customer C Subnet ◄─── eth3            │
│  Shared Services   ◄─── eth4            │
└──────────────────────────────────────────┘
```

## Requirements

### Minimum requirements

- VyOS 1.4 (Sagitta) or newer
- VRF support (FRRouting)
- BGP or static routing for route import/export
- Unique Route Distinguisher (RD) for each VRF

### Network parameters (example)

| VRF | RD | RT Import | RT Export | Networks |
|-----|----|-----------| ----------|----------|
| PRODUCTION | 65000:100 | 65000:100, 65000:999 | 65000:100 | 10.100.0.0/16 |
| DEVELOPMENT | 65000:200 | 65000:200, 65000:999 | 65000:200 | 10.200.0.0/16 |
| SHARED | 65000:999 | 65000:100, 65000:200 | 65000:999 | 10.255.0.0/24 |

## VRF Configuration

### Creating VRF instances

```bash
configure

# VRF PRODUCTION
set vrf name PRODUCTION table '100'
set vrf name PRODUCTION description 'Production environment - Customer A'

# VRF DEVELOPMENT
set vrf name DEVELOPMENT table '200'
set vrf name DEVELOPMENT description 'Development environment - Customer B'

# VRF SHARED
set vrf name SHARED table '999'
set vrf name SHARED description 'Shared services - DNS, NTP, Monitoring'

commit
```

### Assigning interfaces to VRFs

```bash
configure

# eth1 - PRODUCTION
set interfaces ethernet eth1 address '10.100.1.1/24'
set interfaces ethernet eth1 description 'PRODUCTION network'
set interfaces ethernet eth1 vrf 'PRODUCTION'

# eth2 - DEVELOPMENT
set interfaces ethernet eth2 address '10.200.1.1/24'
set interfaces ethernet eth2 description 'DEVELOPMENT network'
set interfaces ethernet eth2 vrf 'DEVELOPMENT'

# eth3 - SHARED
set interfaces ethernet eth3 address '10.255.0.1/24'
set interfaces ethernet eth3 description 'SHARED services'
set interfaces ethernet eth3 vrf 'SHARED'

commit
save
```

## BGP Configuration for Inter-VRF Routing

### BGP base configuration

```bash
configure

# BGP AS number
set protocols bgp system-as '65000'
set protocols bgp parameters router-id '192.168.255.1'

commit
```

### VRF PRODUCTION BGP configuration

```bash
configure

# BGP for VRF PRODUCTION
set protocols bgp address-family ipv4-unicast vrf PRODUCTION rd '65000:100'
set protocols bgp address-family ipv4-unicast vrf PRODUCTION route-target export '65000:100'
set protocols bgp address-family ipv4-unicast vrf PRODUCTION route-target import '65000:100'
set protocols bgp address-family ipv4-unicast vrf PRODUCTION route-target import '65000:999'

# Announce PRODUCTION networks
set protocols bgp address-family ipv4-unicast vrf PRODUCTION network '10.100.0.0/16'

# Redistribute connected
set protocols bgp address-family ipv4-unicast vrf PRODUCTION redistribute connected

commit
```

### VRF DEVELOPMENT BGP configuration

```bash
configure

# BGP for VRF DEVELOPMENT
set protocols bgp address-family ipv4-unicast vrf DEVELOPMENT rd '65000:200'
set protocols bgp address-family ipv4-unicast vrf DEVELOPMENT route-target export '65000:200'
set protocols bgp address-family ipv4-unicast vrf DEVELOPMENT route-target import '65000:200'
set protocols bgp address-family ipv4-unicast vrf DEVELOPMENT route-target import '65000:999'

# Announce DEVELOPMENT networks
set protocols bgp address-family ipv4-unicast vrf DEVELOPMENT network '10.200.0.0/16'
set protocols bgp address-family ipv4-unicast vrf DEVELOPMENT redistribute connected

commit
```

### VRF SHARED BGP configuration

```bash
configure

# BGP for VRF SHARED
set protocols bgp address-family ipv4-unicast vrf SHARED rd '65000:999'
set protocols bgp address-family ipv4-unicast vrf SHARED route-target export '65000:999'
set protocols bgp address-family ipv4-unicast vrf SHARED route-target import '65000:100'
set protocols bgp address-family ipv4-unicast vrf SHARED route-target import '65000:200'

# Announce SHARED services
set protocols bgp address-family ipv4-unicast vrf SHARED network '10.255.0.0/24'
set protocols bgp address-family ipv4-unicast vrf SHARED redistribute connected

commit
save
```

### Understanding Route Targets

**Route Target (RT)** controls the import/export of routes between VRFs:

- **PRODUCTION**:
  - Export `65000:100`: Tag routes with 65000:100
  - Import `65000:100`: Import routes tagged with 65000:100 (its own)
  - Import `65000:999`: Import routes from the SHARED VRF

- **SHARED**:
  - Export `65000:999`: Tag shared services routes
  - Import `65000:100, 65000:200`: Import from PRODUCTION and DEVELOPMENT

**Result**: PRODUCTION sees SHARED but does NOT see DEVELOPMENT (and vice versa).

## Alternative: Static Route Leaking

If BGP is not required, you can use static routes with VRF:

```bash
configure

# In VRF PRODUCTION: route to SHARED via eth3
set vrf name PRODUCTION protocols static route 10.255.0.0/24 next-hop 10.255.0.1 vrf 'SHARED'

# In VRF SHARED: routes back to PRODUCTION and DEVELOPMENT
set vrf name SHARED protocols static route 10.100.0.0/16 next-hop 10.100.1.1 vrf 'PRODUCTION'
set vrf name SHARED protocols static route 10.200.0.0/16 next-hop 10.200.1.1 vrf 'DEVELOPMENT'

commit
save
```

**Note**: Static route leaking is simpler but less flexible than BGP with RT.

## Yandex Cloud Integration

### Scenario: Multi-Tenant Gateway in Yandex Cloud

```bash
configure

# Management interface
set interfaces ethernet eth0 address dhcp
set interfaces ethernet eth0 description 'Yandex Cloud management'

# Customer A (VRF CUSTOMER-A)
set vrf name CUSTOMER-A table '101'
set vrf name CUSTOMER-A description 'Tenant A - E-commerce'

set interfaces ethernet eth1 address '10.1.1.1/24'
set interfaces ethernet eth1 description 'Customer A subnet'
set interfaces ethernet eth1 vrf 'CUSTOMER-A'

# Customer B (VRF CUSTOMER-B)
set vrf name CUSTOMER-B table '102'
set vrf name CUSTOMER-B description 'Tenant B - Analytics'

set interfaces ethernet eth2 address '10.2.1.1/24'
set interfaces ethernet eth2 description 'Customer B subnet'
set interfaces ethernet eth2 vrf 'CUSTOMER-B'

# Shared Services (DNS, NTP, Yandex Monitoring)
set vrf name SHARED-SERVICES table '999'
set vrf name SHARED-SERVICES description 'Shared: DNS, NTP, Monitoring'

set interfaces ethernet eth4 address '10.255.1.1/24'
set interfaces ethernet eth4 description 'Shared services'
set interfaces ethernet eth4 vrf 'SHARED-SERVICES'

# BGP for route leaking
set protocols bgp system-as '65000'
set protocols bgp parameters router-id '10.255.255.1'

# CUSTOMER-A VRF BGP
set protocols bgp address-family ipv4-unicast vrf CUSTOMER-A rd '65000:101'
set protocols bgp address-family ipv4-unicast vrf CUSTOMER-A route-target export '65000:101'
set protocols bgp address-family ipv4-unicast vrf CUSTOMER-A route-target import '65000:101'
set protocols bgp address-family ipv4-unicast vrf CUSTOMER-A route-target import '65000:999'
set protocols bgp address-family ipv4-unicast vrf CUSTOMER-A redistribute connected

# CUSTOMER-B VRF BGP
set protocols bgp address-family ipv4-unicast vrf CUSTOMER-B rd '65000:102'
set protocols bgp address-family ipv4-unicast vrf CUSTOMER-B route-target export '65000:102'
set protocols bgp address-family ipv4-unicast vrf CUSTOMER-B route-target import '65000:102'
set protocols bgp address-family ipv4-unicast vrf CUSTOMER-B route-target import '65000:999'
set protocols bgp address-family ipv4-unicast vrf CUSTOMER-B redistribute connected

# SHARED-SERVICES VRF BGP
set protocols bgp address-family ipv4-unicast vrf SHARED-SERVICES rd '65000:999'
set protocols bgp address-family ipv4-unicast vrf SHARED-SERVICES route-target export '65000:999'
set protocols bgp address-family ipv4-unicast vrf SHARED-SERVICES route-target import '65000:101'
set protocols bgp address-family ipv4-unicast vrf SHARED-SERVICES route-target import '65000:102'
set protocols bgp address-family ipv4-unicast vrf SHARED-SERVICES redistribute connected

commit
save
```

### Yandex Cloud Firewall between VRFs

```bash
configure

# Allow only DNS and NTP from customers to shared
set firewall ipv4-name CUSTOMER-TO-SHARED default-action 'drop'
set firewall ipv4-name CUSTOMER-TO-SHARED rule 10 action 'accept'
set firewall ipv4-name CUSTOMER-TO-SHARED rule 10 protocol 'udp'
set firewall ipv4-name CUSTOMER-TO-SHARED rule 10 destination port '53'
set firewall ipv4-name CUSTOMER-TO-SHARED rule 10 description 'Allow DNS'

set firewall ipv4-name CUSTOMER-TO-SHARED rule 20 action 'accept'
set firewall ipv4-name CUSTOMER-TO-SHARED rule 20 protocol 'udp'
set firewall ipv4-name CUSTOMER-TO-SHARED rule 20 destination port '123'
set firewall ipv4-name CUSTOMER-TO-SHARED rule 20 description 'Allow NTP'

# Apply to VRF
set vrf name CUSTOMER-A ip protocol all export 'CUSTOMER-TO-SHARED'
set vrf name CUSTOMER-B ip protocol all export 'CUSTOMER-TO-SHARED'

commit
save
```

## VK Cloud Integration

### Configuration for VK Cloud

```bash
configure

# VK Cloud management (OpenStack Neutron)
set interfaces ethernet eth0 address dhcp
set interfaces ethernet eth0 description 'VK Cloud management'

# VRF configuration is similar to Yandex Cloud

# VK Cloud specifics:
# 1. Security Groups instead of VyOS firewall (or a combination)
# 2. Floating IP only for management
# 3. MTU 1450 for tenant networks

set interfaces ethernet eth1 mtu '1450'
set interfaces ethernet eth2 mtu '1450'
set interfaces ethernet eth4 mtu '1450'

commit
save
```

### VK Cloud Security Groups

Create Security Groups in the VK Cloud Console for each tenant:

**Customer-A SG**:
- Allow DNS (UDP 53) to Shared Services subnet
- Allow NTP (UDP 123) to Shared Services subnet
- Deny all other to Shared Services

**Customer-B SG**:
- Allow DNS (UDP 53) to Shared Services subnet
- Allow NTP (UDP 123) to Shared Services subnet
- Deny all other to Shared Services

## Configuration Verification

### VRF verification

```bash
# List VRFs
show vrf

# Expected output:
# VRF name          table     description
# --------          -----     -----------
# CUSTOMER-A        101       Tenant A - E-commerce
# CUSTOMER-B        102       Tenant B - Analytics
# SHARED-SERVICES   999       Shared: DNS, NTP, Monitoring

# Details of a specific VRF
show vrf PRODUCTION
```

### Verifying interfaces in a VRF

```bash
# Interfaces by VRF
show interfaces

# IP addresses in a specific VRF
ip vrf exec PRODUCTION ip addr show

# Routing table of a specific VRF
show ip route vrf PRODUCTION
show ip route vrf SHARED
```

### BGP VRF verification

```bash
# BGP summary for all VRFs
show bgp vrf all summary

# BGP routes in a specific VRF
show bgp vrf PRODUCTION ipv4 unicast
show bgp vrf SHARED ipv4 unicast

# Route Distinguisher and Route Target
vtysh
show bgp vrf all
show bgp ipv4 vpn
exit
```

### Route Leaking verification

```bash
# VRF PRODUCTION should see SHARED (10.255.0.0/24)
show ip route vrf PRODUCTION

# Expected output:
# B>* 10.255.0.0/24 [200/0] via 10.255.0.1 (vrf SHARED), 00:10:00

# VRF SHARED should see PRODUCTION and DEVELOPMENT
show ip route vrf SHARED

# Expected output:
# B>* 10.100.0.0/16 [200/0] via 10.100.1.1 (vrf PRODUCTION), 00:10:00
# B>* 10.200.0.0/16 [200/0] via 10.200.1.1 (vrf DEVELOPMENT), 00:10:00
```

### Connectivity test between VRFs

```bash
# Ping from PRODUCTION VRF to SHARED services
ip vrf exec PRODUCTION ping 10.255.0.10 count 4

# Ping from PRODUCTION to DEVELOPMENT (should NOT work - no route leak)
ip vrf exec PRODUCTION ping 10.200.1.10 count 4
# Expected result: "Network is unreachable"

# Traceroute into another VRF
ip vrf exec PRODUCTION traceroute 10.255.0.10
```

### DNS Test (if DNS is in the SHARED VRF)

```bash
# DNS server in SHARED VRF: 10.255.0.53

# Test from PRODUCTION VRF
ip vrf exec PRODUCTION nslookup example.com 10.255.0.53
```

## Troubleshooting

### Problem 1: A VRF does not see routes from another VRF

**Symptoms**:
```bash
show ip route vrf PRODUCTION
# No routes from the SHARED VRF
```

**Causes and solutions**:

1. **Incorrect Route Target**:
   ```bash
   # Check RT configuration
   vtysh
   show running-config | grep "route-target"

   # PRODUCTION must import 65000:999
   # SHARED must export 65000:999

   # Fix
   configure
   set protocols bgp address-family ipv4-unicast vrf PRODUCTION route-target import '65000:999'
   commit
   ```

2. **BGP not running for the VRF**:
   ```bash
   show bgp vrf PRODUCTION summary
   # Should show the BGP process

   # If not, check the configuration
   show configuration commands | grep "vrf PRODUCTION"
   ```

3. **No network announcement**:
   ```bash
   vtysh
   show bgp vrf SHARED ipv4 unicast

   # There should be routes to announce
   # Add a network statement
   configure
   set protocols bgp address-family ipv4-unicast vrf SHARED network '10.255.0.0/24'
   commit
   ```

### Problem 2: Interface is not assigned to a VRF

**Symptoms**:
```bash
show interfaces ethernet eth1
# No VRF in the output
```

**Solution**:
```bash
configure
set interfaces ethernet eth1 vrf 'PRODUCTION'
commit

# IMPORTANT: After VRF assignment the interface will go down/up
# All IP addresses are preserved
```

### Problem 3: IP overlap between VRFs causes issues

**Symptoms**:
CUSTOMER-A and CUSTOMER-B use 10.0.0.0/8, and packets go to the wrong place

**Solution**:
This is normal for VRFs, but proper configuration is required:

```bash
# Verify that interfaces are in the correct VRFs
show vrf

# Ensure that routing table numbers are unique
show configuration commands | grep "table"

# ALWAYS test connectivity by specifying the VRF
ip vrf exec CUSTOMER-A ping 10.0.0.1
ip vrf exec CUSTOMER-B ping 10.0.0.1
# Different destinations, despite the identical IP
```

### Problem 4: Performance issues

**Symptoms**:
Slow throughput between VRFs

**Causes**:
- Routing through the kernel, not hardware offload
- Firewall processing

**Solution**:
```bash
# Check firewall rules
show firewall statistics

# Simplify firewall rules
# Use connection tracking
set firewall ipv4-name VRF-TO-VRF rule 1 action 'accept'
set firewall ipv4-name VRF-TO-VRF rule 1 state established 'enable'
set firewall ipv4-name VRF-TO-VRF rule 1 state related 'enable'
commit
```

## Best Practices

### 1. VRF Naming Convention

```bash
# Use descriptive names
VRF PRODUCTION        # Not VRF-1
VRF CUSTOMER-ACME     # Not VRF-CUST-001
VRF SHARED-SERVICES   # Not VRF-SHARED
```

### 2. Route Distinguisher (RD) Planning

```bash
# Format: AS:NN
# AS = your AS number (or 65000 for private)
# NN = unique VRF number

# Numbering scheme:
# 65000:100-199  = Production VRFs
# 65000:200-299  = Development VRFs
# 65000:300-399  = Testing VRFs
# 65000:900-999  = Shared/Management VRFs
```

### 3. Route Target Design

**Hub-and-Spoke** (our example):
```
PRODUCTION → SHARED (import 999)
DEVELOPMENT → SHARED (import 999)
SHARED → ALL (import 100, 200)
```

**Full Mesh** (all VRFs see each other):
```
ALL VRF → (import * export *)
```

**Partial Mesh**:
Use route-maps to filter specific prefixes.

### 4. Security between VRFs

```bash
# Use a firewall even between VRFs
set firewall ipv4-name VRF-ISOLATION default-action 'drop'
set firewall ipv4-name VRF-ISOLATION rule 10 action 'accept'
set firewall ipv4-name VRF-ISOLATION rule 10 destination address '10.255.0.53'
set firewall ipv4-name VRF-ISOLATION rule 10 destination port '53'
set firewall ipv4-name VRF-ISOLATION rule 10 protocol 'udp'

# Apply
set vrf name PRODUCTION ip protocol all export 'VRF-ISOLATION'
commit
```

### 5. Logging and Monitoring

```bash
# BGP logging
vtysh
conf t
log syslog informational
bgp log-neighbor-changes
exit
exit

# Monitor VRF routing tables
watch -n 5 'show ip route vrf PRODUCTION; show ip route vrf SHARED'

# BGP VRF monitoring
watch -n 5 'show bgp vrf all summary'
```

### 6. Documentation

Document the VRF topology:

```yaml
# /etc/vyos/vrf-topology.yaml
vrf_topology:
  PRODUCTION:
    table: 100
    rd: "65000:100"
    rt_export: "65000:100"
    rt_import: ["65000:100", "65000:999"]
    interfaces: ["eth1"]
    networks: ["10.100.0.0/16"]
    access_to: ["SHARED"]

  SHARED:
    table: 999
    rd: "65000:999"
    rt_export: "65000:999"
    rt_import: ["65000:100", "65000:200"]
    interfaces: ["eth3"]
    networks: ["10.255.0.0/24"]
    services: ["DNS: 10.255.0.53", "NTP: 10.255.0.123"]
```

## Advanced Configuration

### Route-Maps for Selective Filtering

```bash
configure

# Prefix-list: allow only the DNS server
set policy prefix-list ALLOW-DNS-ONLY rule 10 action 'permit'
set policy prefix-list ALLOW-DNS-ONLY rule 10 prefix '10.255.0.53/32'

# Route-map: apply the prefix-list
set policy route-map FILTER-SHARED-IMPORT rule 10 action 'permit'
set policy route-map FILTER-SHARED-IMPORT rule 10 match ip address prefix-list 'ALLOW-DNS-ONLY'

# Apply the route-map to BGP import
# (requires FRR vtysh)
vtysh
conf t
router bgp 65000
address-family ipv4 unicast vrf PRODUCTION
import vrf route-map FILTER-SHARED-IMPORT
exit
exit
exit
exit
```

### VRF with MPLS (for L3VPN)

```bash
# Enable MPLS on interfaces
set interfaces ethernet eth0 mpls

# LDP
set protocols mpls ldp interface 'eth0'
set protocols mpls ldp router-id '192.168.255.1'

# BGP VPNv4
vtysh
conf t
router bgp 65000
address-family vpnv4 unicast
neighbor 192.168.255.2 activate
exit
exit
exit
```

## References

- [VyOS VRF Configuration](https://docs.vyos.io/en/latest/configuration/vrf/index.html)
- [RFC 4364 - BGP/MPLS IP VPNs](https://datatracker.ietf.org/doc/html/rfc4364)
- [RFC 4659 - BGP-MPLS IP VPN Extension for IPv6](https://datatracker.ietf.org/doc/html/rfc4659)
- [FRRouting VRF Documentation](http://docs.frrouting.org/en/latest/vrf.html)
- [Yandex Cloud VPC](https://cloud.yandex.ru/docs/vpc/)
- [VK Cloud Multi-Tenancy](https://mcs.mail.ru/docs/)