# MPLS - Multiprotocol Label Switching

> Configure MPLS on VyOS: MPLS architecture, LDP, L3VPN, RSVP-TE, traffic engineering, and label switching for carrier and enterprise networks

Source: https://opennix.org/en/docs/vyos/routing/vyos-mpls/


MPLS (Multiprotocol Label Switching) is a label-based packet switching technology that delivers high performance, traffic engineering, and VPN services.

## Overview

MPLS is a packet forwarding paradigm that uses short labels (32-bit) instead of long IP addresses to make routing decisions.

### What is MPLS

**Core concepts**:
- **Label Switching** - forwarding based on labels instead of an IP lookup
- **LSP (Label Switched Path)** - a unidirectional path across the MPLS network
- **Label Stack** - a hierarchical structure of labels (used for tunneling)
- **FEC (Forwarding Equivalence Class)** - a group of packets that receive identical treatment

**Architecture**:
```
[Ingress LER] ----> [LSR] ----> [LSR] ----> [Egress LER]
   (Push)           (Swap)      (Swap)        (Pop)
    |                |            |              |
    v                v            v              v
 IP packet      Label=100    Label=200      IP packet
 + Label=100                               (no label)
```

### MPLS network components

**Label Edge Router (LER)**:
- Border router of the MPLS network
- Ingress LER: adds the label (push)
- Egress LER: removes the label (pop)
- Performs the IP lookup at the edge

**Label Switch Router (LSR)**:
- Internal router of the MPLS network
- Performs label switching only (swap)
- Does not inspect the IP header
- High performance

**Control Plane**:
- LDP (Label Distribution Protocol)
- RSVP-TE (Traffic Engineering)
- BGP (for L3VPN)
- IGP (OSPF, IS-IS) for basic routing

**Data Plane**:
- Label Forwarding Information Base (LFIB)
- Label Information Base (LIB)
- Label-based forwarding plane

### MPLS header

Structure of the MPLS header (32 bits):

```
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                Label                  | TC  |S|       TTL     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
```

**Fields**:
- **Label** (20 bits) - the label value (0-1048575)
- **TC/EXP** (3 bits) - Traffic Class (QoS)
- **S** (1 bit) - Bottom of Stack (the last label in the stack)
- **TTL** (8 bits) - Time to Live

**Reserved labels**:
- **0** - IPv4 Explicit NULL
- **1** - Router Alert
- **2** - IPv6 Explicit NULL
- **3** - Implicit NULL
- **4-15** - Reserved
- **16-1048575** - Available for use

### Benefits of MPLS

**Performance**:
- Fast label-based switching (vs IP lookup)
- Hardware acceleration of label switching
- Reduced CPU load

**Traffic Engineering**:
- Explicit paths
- Bandwidth reservation
- Fast Reroute for resilience
- Load balancing across multiple paths

**VPN Services**:
- L2VPN (VPLS, VPWS)
- L3VPN (MPLS VPN)
- Traffic isolation between different customers
- Scalability (thousands of VPNs)

**QoS**:
- Traffic Class in the MPLS header
- DiffServ integration
- Guaranteed bandwidth

**Applications**:
- Service Provider networks
- Enterprise WAN
- Data Center Interconnect (DCI)
- 5G Transport Network

### Current VyOS limitations

**Implemented**:
- LDP (Label Distribution Protocol)
- MPLS on interfaces
- Label switching (basic)
- Integration with OSPF/BGP

**Not implemented**:
- MPLS L2VPN (VPLS, VPWS)
- MPLS L3VPN (VRF-lite is supported separately)
- RSVP-TE (Traffic Engineering)
- mVPN (Multicast VPN)
- MPLS OAM (LSP Ping, Traceroute)

**Status**: MPLS support in VyOS is still evolving. Basic LDP works reliably, but advanced features are not yet available.

## Label Distribution Protocol (LDP)

LDP is the protocol for automatically distributing labels between MPLS routers.

### LDP architecture

**How it works**:
1. LSRs discover each other (hello messages)
2. A TCP session is established (port 646)
3. Label bindings are exchanged (FEC <-> Label)
4. The LFIB is built on each LSR

**Message types**:
- **Discovery** (UDP 646) - hello messages
- **Session** (TCP 646) - session establishment
- **Advertisement** - label distribution
- **Notification** - errors and events

**Label Distribution Mode**:
- **Downstream Unsolicited** - the LSR advertises labels without a request
- **Downstream on Demand** - labels are advertised on request
- VyOS uses Downstream Unsolicited (default)

**Label Retention Mode**:
- **Liberal** - retain labels from all LSRs
- **Conservative** - retain labels only from the next-hop
- VyOS uses Liberal mode

### Basic LDP configuration

**Minimal configuration**:

```
# Enable MPLS on interface
set protocols mpls interface eth1

# Configure LDP
set protocols mpls ldp interface eth1
set protocols mpls ldp router-id 10.255.255.1

commit
save
```

**Full configuration**:

```
# MPLS interfaces
set protocols mpls interface eth1
set protocols mpls interface eth2

# LDP configuration
set protocols mpls ldp router-id 10.255.255.1
set protocols mpls ldp discovery transport-ipv4-address 10.255.255.1

# LDP on interfaces
set protocols mpls ldp interface eth1
set protocols mpls ldp interface eth2

# Optional: IGP for reachability
set protocols ospf area 0 network 10.255.255.1/32
set protocols ospf area 0 network 10.0.0.0/24
set protocols ospf area 0 network 10.0.1.0/24

commit
save
```

**Parameters**:
- `router-id` - unique LSR identifier (required)
- `transport-ipv4-address` - address used for TCP sessions
- `interface` - interfaces with LDP enabled

### LDP Router ID

The Router ID is the unique identifier of the LDP router (IPv4 format).

```
set protocols mpls ldp router-id 10.255.255.1
commit
```

**Router ID selection**:
1. An explicitly configured router-id
2. The highest loopback interface IP
3. The highest physical interface IP

**Recommendation**: Use a loopback address for stability.

### Transport Address

The address used to establish TCP sessions between LSRs.

```
set protocols mpls ldp discovery transport-ipv4-address 10.255.255.1
commit
```

**Usage**:
- Usually matches the router-id
- Must be reachable through the IGP
- Use a loopback address for reliability

### LDP Interfaces

Enabling LDP on interfaces:

```
set protocols mpls ldp interface eth1
set protocols mpls ldp interface eth2
set protocols mpls ldp interface eth3
commit
```

**Hello Parameters**:

```
# Hello interval (default: 5 seconds)
set protocols mpls ldp discovery hello-interval 5

# Hold time (default: 15 seconds)
set protocols mpls ldp discovery hello-holdtime 15

commit
```

**Targeted LDP**:

For LDP sessions with LSRs that are not directly connected:

```
set protocols mpls ldp discovery targeted-hello accept
set protocols mpls ldp discovery targeted-hello peer-ipv4-address 10.255.255.10
commit
```

### LDP Session Parameters

**Keepalive and Hold Time**:

```
set protocols mpls ldp session holdtime 180
set protocols mpls ldp session keepalive-interval 60
commit
```

Defaults:
- Keepalive: 60 seconds
- Hold time: 180 seconds

**MD5 Authentication**:

```
set protocols mpls ldp neighbor 10.255.255.2 password 'SecureLDP123!'
commit
```

Protects the LDP session against spoofing.

### LDP Label Allocation

**Allocation Mode**:

By default VyOS uses per-platform label allocation:
- A single label for all next-hops toward a prefix
- More efficient use of the label space

**Label Range**:

```
set protocols mpls label-range dynamic-range-start 16
set protocols mpls label-range dynamic-range-end 1048575
commit
```

The range of labels used for dynamic allocation.

## MPLS Forwarding

### MPLS on interfaces

Enabling MPLS on an interface:

```
set protocols mpls interface eth1
commit
```

**Effect**:
- The interface starts processing MPLS packets
- Label switching is enabled
- The interface participates in building LSPs

**Multiple interfaces**:

```
set protocols mpls interface eth1
set protocols mpls interface eth2
set protocols mpls interface eth3
set protocols mpls interface bond0
commit
```

### Label Operations

**PUSH** (adding a label):
- On the Ingress LER
- The IP packet receives an MPLS label
- Performed based on the FEC

**SWAP** (replacing a label):
- On intermediate LSRs
- The incoming label is replaced with an outgoing label
- Lookup in the LFIB

**POP** (removing a label):
- On the Egress LER
- The label is removed
- The packet is processed as IP
- Penultimate Hop Popping (PHP): pop on the penultimate LSR

### PHP (Penultimate Hop Popping)

By default LDP uses PHP:

```
[R1] --100--> [R2] --200--> [R3] --IP--> [R4]
Ingress       LSR          Penultimate   Egress
```

**Benefits**:
- Reduced load on the Egress LER
- A single action instead of two (pop + IP lookup)

**Explicit NULL**:

Disable PHP (use an explicit null label):

```
set protocols mpls ldp allocation ipv4 explicit-null
commit
```

The Egress LER will receive the packet with label 0 (Explicit NULL).

### TTL Propagation

Copying the TTL between the IP and MPLS headers:

```
# Enable TTL propagation (default)
set protocols mpls ttl-propagation enable
commit

# Disable TTL propagation (hide the MPLS topology)
set protocols mpls ttl-propagation disable
commit
```

**Disable**: Hides the number of LSRs in the MPLS network from traceroute.

## L3VPN with MPLS

MPLS L3VPN (RFC 4364) is an MPLS-based VPN service for isolating IP traffic.

### L3VPN architecture

**Components**:
- **CE (Customer Edge)** - the customer's router
- **PE (Provider Edge)** - the provider's border router
- **P (Provider)** - the provider's internal router

**Technologies**:
- **VRF (Virtual Routing and Forwarding)** - isolated routing tables
- **Route Distinguisher (RD)** - prefix uniqueness
- **Route Target (RT)** - import/export between VRFs
- **MP-BGP** - distribution of VPN routes
- **MPLS** - encapsulation for transport across the network

### VRF Configuration

VyOS supports VRF, but integration with MPLS L3VPN is limited.

**Creating a VRF**:

```
# VRF for customer A
set vrf name CUSTOMER-A table 100
set vrf name CUSTOMER-A description 'Customer A VPN'

# VRF for customer B
set vrf name CUSTOMER-B table 101
set vrf name CUSTOMER-B description 'Customer B VPN'

commit
```

**Binding an interface to a VRF**:

```
set interfaces ethernet eth2 vrf CUSTOMER-A
set interfaces ethernet eth2 address 192.168.10.1/24

set interfaces ethernet eth3 vrf CUSTOMER-B
set interfaces ethernet eth3 address 192.168.10.1/24

commit
```

Identical IP addresses in different VRFs do not conflict.

### BGP for L3VPN

**BGP inside a VRF**:

```
# Global BGP
set protocols bgp system-as 65000

# BGP in VRF CUSTOMER-A
set vrf name CUSTOMER-A protocols bgp system-as 65000
set vrf name CUSTOMER-A protocols bgp neighbor 192.168.10.2 remote-as 65001
set vrf name CUSTOMER-A protocols bgp address-family ipv4-unicast network 10.1.0.0/16

# BGP in VRF CUSTOMER-B
set vrf name CUSTOMER-B protocols bgp system-as 65000
set vrf name CUSTOMER-B protocols bgp neighbor 192.168.10.2 remote-as 65002
set vrf name CUSTOMER-B protocols bgp address-family ipv4-unicast network 10.2.0.0/16

commit
```

### VyOS L3VPN limitations

**Current status**:
- VRF is implemented and works
- MPLS LDP works separately
- No VRF + MPLS integration
- No MP-BGP for VPNv4/VPNv6
- No Route Distinguisher/Target

**Workaround**:
- Use VRF without MPLS (VLAN-based isolation)
- Use IPsec/GRE/VXLAN for an overlay VPN
- Wait for a full MPLS L3VPN implementation in future versions

### Example L3VPN architecture

Conceptual diagram (for understanding; a full implementation in VyOS is pending):

```
[CE1] --eBGP-- [PE1] ====MPLS/LDP==== [PE2] --eBGP-- [CE2]
 AS65001      AS65000    P Routers     AS65000      AS65002
              VRF-A                    VRF-A
              RD:65000:1              RD:65000:1
              RT:65000:1              RT:65000:1
```

**PE1 (theoretical configuration)**:

```
# VRF
set vrf name VPN-A table 100
set vrf name VPN-A rd 65000:1
set vrf name VPN-A rt import 65000:1
set vrf name VPN-A rt export 65000:1

# CE-facing interface
set interfaces ethernet eth2 vrf VPN-A
set interfaces ethernet eth2 address 10.0.1.1/30

# BGP with CE
set vrf name VPN-A protocols bgp system-as 65000
set vrf name VPN-A protocols bgp neighbor 10.0.1.2 remote-as 65001

# MP-BGP with other PE
set protocols bgp system-as 65000
set protocols bgp neighbor 10.255.255.2 remote-as 65000
set protocols bgp address-family l3vpn-ipv4

# MPLS/LDP
set protocols mpls interface eth1
set protocols mpls ldp interface eth1
set protocols mpls ldp router-id 10.255.255.1

commit
```

**Note**: This is a conceptual configuration; not all parameters are implemented in the current VyOS version.

## RSVP-TE (Traffic Engineering)

RSVP-TE (RFC 3209) is an extension of RSVP for reserving resources and specifying explicit paths.

### RSVP-TE concept

**Capabilities**:
- Explicit path routing (bypassing automatic IGP paths)
- Bandwidth reservation (guaranteed bandwidth)
- Fast Reroute (protection against failures)
- Constraint-based routing

**Comparison with LDP**:

| Feature | LDP | RSVP-TE |
|---------|-----|---------|
| Path selection | Follow IGP | Explicit path |
| Bandwidth | No reservation | Reservation |
| Protection | No | Fast Reroute |
| Complexity | Simple | Complex |
| Overhead | Low | High |

### RSVP-TE in VyOS

**Current status**: RSVP-TE is NOT implemented in VyOS.

**Alternatives**:
- Use LDP for basic MPLS
- Policy-based routing for traffic steering
- Static LSPs (if implemented)

### Conceptual RSVP-TE configuration

An example of what the configuration might look like (NOT IMPLEMENTED):

```
# Enable RSVP-TE
set protocols mpls rsvp-te enable

# Interface bandwidth
set protocols mpls rsvp-te interface eth1 bandwidth 1000000
set protocols mpls rsvp-te interface eth2 bandwidth 1000000

# LSP tunnel
set protocols mpls rsvp-te tunnel LSP-TO-R5
set protocols mpls rsvp-te tunnel LSP-TO-R5 destination 10.255.255.5
set protocols mpls rsvp-te tunnel LSP-TO-R5 bandwidth 100000
set protocols mpls rsvp-te tunnel LSP-TO-R5 priority setup 7
set protocols mpls rsvp-te tunnel LSP-TO-R5 priority hold 7

# Explicit path
set protocols mpls rsvp-te path PATH-VIA-R2
set protocols mpls rsvp-te path PATH-VIA-R2 hop 10 type strict
set protocols mpls rsvp-te path PATH-VIA-R2 hop 10 address 10.0.1.2
set protocols mpls rsvp-te path PATH-VIA-R2 hop 20 type strict
set protocols mpls rsvp-te path PATH-VIA-R2 hop 20 address 10.0.2.5

set protocols mpls rsvp-te tunnel LSP-TO-R5 explicit-path PATH-VIA-R2

commit
```

**Fast Reroute**:

```
set protocols mpls rsvp-te tunnel LSP-TO-R5 fast-reroute enable
set protocols mpls rsvp-te tunnel LSP-TO-R5 fast-reroute link-protection
commit
```

## Configuration examples

### Simple MPLS network (3 routers)

**Topology**:

```
[R1] eth1 <----> eth1 [R2] eth2 <----> eth1 [R3]
10.0.0.0/30        10.0.0.4/30
.1            .2   .5            .6
```

**R1 (Ingress LER)**:

```
# Interfaces
set interfaces ethernet eth1 address 10.0.0.1/30
set interfaces loopback lo address 10.255.255.1/32

# OSPF
set protocols ospf area 0 network 10.0.0.0/30
set protocols ospf area 0 network 10.255.255.1/32
set protocols ospf parameters router-id 10.255.255.1

# MPLS/LDP
set protocols mpls interface eth1
set protocols mpls ldp interface eth1
set protocols mpls ldp router-id 10.255.255.1
set protocols mpls ldp discovery transport-ipv4-address 10.255.255.1

commit
save
```

**R2 (LSR)**:

```
# Interfaces
set interfaces ethernet eth1 address 10.0.0.2/30
set interfaces ethernet eth2 address 10.0.0.5/30
set interfaces loopback lo address 10.255.255.2/32

# OSPF
set protocols ospf area 0 network 10.0.0.0/30
set protocols ospf area 0 network 10.0.0.4/30
set protocols ospf area 0 network 10.255.255.2/32
set protocols ospf parameters router-id 10.255.255.2

# MPLS/LDP
set protocols mpls interface eth1
set protocols mpls interface eth2
set protocols mpls ldp interface eth1
set protocols mpls ldp interface eth2
set protocols mpls ldp router-id 10.255.255.2
set protocols mpls ldp discovery transport-ipv4-address 10.255.255.2

commit
save
```

**R3 (Egress LER)**:

```
# Interfaces
set interfaces ethernet eth1 address 10.0.0.6/30
set interfaces loopback lo address 10.255.255.3/32

# OSPF
set protocols ospf area 0 network 10.0.0.4/30
set protocols ospf area 0 network 10.255.255.3/32
set protocols ospf parameters router-id 10.255.255.3

# MPLS/LDP
set protocols mpls interface eth1
set protocols mpls ldp interface eth1
set protocols mpls ldp router-id 10.255.255.3
set protocols mpls ldp discovery transport-ipv4-address 10.255.255.3

commit
save
```

### MPLS with BGP (for an L3VPN backbone)

**R1 (PE Router)**:

```
# Interfaces
set interfaces ethernet eth1 address 10.0.0.1/30
set interfaces ethernet eth2 address 192.168.1.1/24
set interfaces loopback lo address 10.255.255.1/32

# OSPF for the MPLS core
set protocols ospf area 0 network 10.0.0.0/30
set protocols ospf area 0 network 10.255.255.1/32
set protocols ospf parameters router-id 10.255.255.1

# MPLS/LDP
set protocols mpls interface eth1
set protocols mpls ldp interface eth1
set protocols mpls ldp router-id 10.255.255.1
set protocols mpls ldp discovery transport-ipv4-address 10.255.255.1

# iBGP with other PEs
set protocols bgp system-as 65000
set protocols bgp parameters router-id 10.255.255.1
set protocols bgp neighbor 10.255.255.3 remote-as 65000
set protocols bgp neighbor 10.255.255.3 update-source 10.255.255.1
set protocols bgp neighbor 10.255.255.3 address-family ipv4-unicast

# Customer network
set protocols bgp address-family ipv4-unicast network 192.168.1.0/24

commit
save
```

### MPLS for a Service Provider

**Topology**:

```
                    MPLS Core
         ┌─────────────────────────────┐
         │                             │
[CE-A]--[PE1]--[P1]--[P2]--[PE2]--[CE-B]
         │                             │
         └─────────────────────────────┘
              LDP + OSPF/IS-IS
```

**PE1 (Provider Edge)**:

```
# Core-facing interfaces
set interfaces ethernet eth1 address 10.0.1.1/30
set interfaces loopback lo address 10.255.255.1/32

# Customer-facing interface
set interfaces ethernet eth2 address 192.168.10.1/30

# IGP (OSPF) for the MPLS core
set protocols ospf area 0 network 10.0.1.0/30
set protocols ospf area 0 network 10.255.255.1/32
set protocols ospf parameters router-id 10.255.255.1

# MPLS/LDP on core interfaces
set protocols mpls interface eth1
set protocols mpls ldp interface eth1
set protocols mpls ldp router-id 10.255.255.1
set protocols mpls ldp discovery transport-ipv4-address 10.255.255.1

# BGP with the customer (CE)
set protocols bgp system-as 65000
set protocols bgp neighbor 192.168.10.2 remote-as 65001
set protocols bgp neighbor 192.168.10.2 address-family ipv4-unicast

# iBGP with other PEs to exchange customer routes
set protocols bgp neighbor 10.255.255.2 remote-as 65000
set protocols bgp neighbor 10.255.255.2 update-source 10.255.255.1
set protocols bgp neighbor 10.255.255.2 address-family ipv4-unicast

commit
save
```

**P1 (Provider Core)**:

```
# Core interfaces
set interfaces ethernet eth1 address 10.0.1.2/30
set interfaces ethernet eth2 address 10.0.2.1/30
set interfaces loopback lo address 10.255.255.10/32

# OSPF
set protocols ospf area 0 network 10.0.1.0/30
set protocols ospf area 0 network 10.0.2.0/30
set protocols ospf area 0 network 10.255.255.10/32
set protocols ospf parameters router-id 10.255.255.10

# MPLS/LDP on all core interfaces
set protocols mpls interface eth1
set protocols mpls interface eth2
set protocols mpls ldp interface eth1
set protocols mpls ldp interface eth2
set protocols mpls ldp router-id 10.255.255.10
set protocols mpls ldp discovery transport-ipv4-address 10.255.255.10

commit
save
```

### Yandex Cloud example: MPLS L3VPN for Enterprise

**Scenario**: A large enterprise with offices in Moscow and Saint Petersburg uses Yandex Cloud for MPLS L3VPN.

**Architecture**:

```
Moscow (HQ)                 Yandex Cloud                St. Petersburg (Branch)
[CE-MSK]────────[PE-MSK]════════════[PE-SPB]────────[CE-SPB]
192.168.1.0/24   MPLS Core (VRF: CUSTOMER-A)    192.168.2.0/24
AS 65001         AS 65000                        AS 65001
```

**PE-MSK (Yandex Cloud - Moscow)**:

```
# Core interface
set interfaces ethernet eth1 address 10.100.0.1/30
set interfaces loopback lo address 10.255.0.1/32

# Customer interface
set interfaces ethernet eth2 address 10.1.0.1/30

# OSPF in the MPLS core
set protocols ospf area 0 network 10.100.0.0/30
set protocols ospf area 0 network 10.255.0.1/32
set protocols ospf parameters router-id 10.255.0.1

# MPLS/LDP
set protocols mpls interface eth1
set protocols mpls ldp interface eth1
set protocols mpls ldp router-id 10.255.0.1
set protocols mpls ldp discovery transport-ipv4-address 10.255.0.1

# VRF for the customer
set vrf name CUSTOMER-A table 100
set vrf name CUSTOMER-A description 'Enterprise Customer A'
set interfaces ethernet eth2 vrf CUSTOMER-A

# BGP with CE (inside the VRF)
set vrf name CUSTOMER-A protocols bgp system-as 65000
set vrf name CUSTOMER-A protocols bgp neighbor 10.1.0.2 remote-as 65001
set vrf name CUSTOMER-A protocols bgp neighbor 10.1.0.2 address-family ipv4-unicast

# iBGP with PE-SPB to exchange VPN routes
set protocols bgp system-as 65000
set protocols bgp parameters router-id 10.255.0.1
set protocols bgp neighbor 10.255.0.2 remote-as 65000
set protocols bgp neighbor 10.255.0.2 update-source 10.255.0.1
set protocols bgp neighbor 10.255.0.2 address-family ipv4-unicast

# Static route for the customer prefix (alternative to BGP)
set vrf name CUSTOMER-A protocols static route 192.168.2.0/24 next-hop 10.1.0.2

commit
save
```

**CE-MSK (Customer - Moscow HQ)**:

```
# WAN interface (toward PE)
set interfaces ethernet eth0 address 10.1.0.2/30

# LAN interface
set interfaces ethernet eth1 address 192.168.1.1/24

# BGP with PE
set protocols bgp system-as 65001
set protocols bgp parameters router-id 192.168.1.1
set protocols bgp neighbor 10.1.0.1 remote-as 65000
set protocols bgp neighbor 10.1.0.1 address-family ipv4-unicast

# Announce local network
set protocols bgp address-family ipv4-unicast network 192.168.1.0/24

# Default route via PE
set protocols static route 0.0.0.0/0 next-hop 10.1.0.1

commit
save
```

**Result**: The Moscow and Saint Petersburg offices are connected through Yandex Cloud MPLS L3VPN with guaranteed traffic isolation.

### VK Cloud example: MPLS Backbone for an ISP

**Scenario**: A regional ISP uses VK Cloud to build an MPLS backbone between cities.

**Architecture**:

```
City A               VK Cloud MPLS Core           City B
[BRAS-A]────[PE-A]═══[P1]═══[P2]═══[PE-B]────[BRAS-B]
Customers    LDP             LDP         LDP    Customers
```

**PE-A (VK Cloud - Edge in City A)**:

```
# Interfaces
set interfaces ethernet eth1 address 10.200.1.1/30
set interfaces ethernet eth2 address 10.200.1.5/30
set interfaces ethernet eth3 address 100.64.1.1/24
set interfaces loopback lo address 10.255.1.1/32

# OSPF
set protocols ospf area 0 network 10.200.1.0/30
set protocols ospf area 0 network 10.200.1.4/30
set protocols ospf area 0 network 10.255.1.1/32
set protocols ospf parameters router-id 10.255.1.1

# MPLS/LDP on core interfaces
set protocols mpls interface eth1
set protocols mpls interface eth2
set protocols mpls ldp interface eth1
set protocols mpls ldp interface eth2
set protocols mpls ldp router-id 10.255.1.1
set protocols mpls ldp discovery transport-ipv4-address 10.255.1.1

# BGP for customer routes (iBGP)
set protocols bgp system-as 64512
set protocols bgp parameters router-id 10.255.1.1

# iBGP with other PEs
set protocols bgp neighbor 10.255.1.2 remote-as 64512
set protocols bgp neighbor 10.255.1.2 update-source 10.255.1.1
set protocols bgp neighbor 10.255.1.2 address-family ipv4-unicast
set protocols bgp neighbor 10.255.1.2 address-family ipv4-unicast next-hop-self

# BRAS connection
set protocols bgp neighbor 100.64.1.10 remote-as 64512
set protocols bgp neighbor 100.64.1.10 address-family ipv4-unicast

# Redistribute connected for customer subnets
set protocols bgp address-family ipv4-unicast redistribute connected route-map CUSTOMERS

# Route-map
set policy route-map CUSTOMERS rule 10 action permit
set policy route-map CUSTOMERS rule 10 match interface eth3

commit
save
```

**P1 (VK Cloud - Core Router)**:

```
# Interfaces
set interfaces ethernet eth1 address 10.200.1.2/30
set interfaces ethernet eth2 address 10.200.2.1/30
set interfaces ethernet eth3 address 10.200.3.1/30
set interfaces loopback lo address 10.255.100.1/32

# OSPF with high priority (for stability)
set protocols ospf area 0 network 10.200.1.0/30
set protocols ospf area 0 network 10.200.2.0/30
set protocols ospf area 0 network 10.200.3.0/30
set protocols ospf area 0 network 10.255.100.1/32
set protocols ospf parameters router-id 10.255.100.1

# MPLS/LDP on all interfaces
set protocols mpls interface eth1
set protocols mpls interface eth2
set protocols mpls interface eth3
set protocols mpls ldp interface eth1
set protocols mpls ldp interface eth2
set protocols mpls ldp interface eth3
set protocols mpls ldp router-id 10.255.100.1
set protocols mpls ldp discovery transport-ipv4-address 10.255.100.1

# NO BGP on the core router (MPLS transit only)

commit
save
```

**Benefits for the ISP**:
- High performance (label switching)
- Scalability (thousands of customer routes)
- Traffic Engineering capabilities
- Fast convergence with BFD

## Operational commands

### Checking MPLS

**MPLS interfaces**:

```
show mpls interface
```

Output:
```
Interface          State  MPLS Enabled
eth1              up     yes
eth2              up     yes
```

**MPLS table**:

```
show mpls table
```

Output:
```
Inbound Label  Type        Nexthop       Outbound Label
16             LDP         10.0.0.2      17
17             LDP         10.0.0.2      implicit-null
18             LDP         10.0.0.2      19
```

### LDP Neighbors

**Neighbor list**:

```
show mpls ldp neighbor
```

Output:
```
Peer LDP Ident: 10.255.255.2:0
  TCP connection: 10.255.255.2:646 - 10.255.255.1:37521
  State: OPERATIONAL
  Up time: 01:23:45
  LDP Discovery Sources:
    Interface: eth1

Peer LDP Ident: 10.255.255.3:0
  TCP connection: 10.255.255.3:646 - 10.255.255.1:40123
  State: OPERATIONAL
  Up time: 02:15:30
  LDP Discovery Sources:
    Interface: eth2
```

**Details for a specific neighbor**:

```
show mpls ldp neighbor 10.255.255.2 detail
```

### LDP Bindings

**Label bindings**:

```
show mpls ldp binding
```

Output:
```
Destination               Nexthop           Local Label    Remote Label
10.255.255.1/32          10.0.0.1          imp-null       -
10.255.255.2/32          10.0.0.2          16             imp-null
10.255.255.3/32          10.0.0.2          17             18
192.168.1.0/24           10.0.0.2          18             20
```

**Bindings for a specific prefix**:

```
show mpls ldp binding 192.168.1.0/24
```

### LDP Discovery

**Discovery information**:

```
show mpls ldp discovery
```

Output:
```
Local LDP Identifier: 10.255.255.1:0
Discovery Sources:
  Interface: eth1
    Transport Address: 10.255.255.1
    Hello Interval: 5
    Hello Holdtime: 15
    Link Hellos Sent: 1234
    Link Hellos Received: 1230
```

### Debug and logs

**Enable debug**:

```
# Debug LDP events
debug mpls ldp event

# Debug LDP messages
debug mpls ldp messages

# Debug LDP zebra
debug mpls ldp zebra

# View logs
show log | match mpls
```

**Disable debug**:

```
no debug mpls ldp event
no debug mpls ldp messages
no debug mpls ldp zebra
```

### Forwarding Table

**Kernel forwarding table**:

```
show ip route
```

MPLS routes are marked as:
```
L   192.168.1.0/24 [110/20] via 10.0.0.2, eth1, label 20, 01:23:45
```

**FIB (Forwarding Information Base)**:

```
show ip fib
```

## Troubleshooting

### LDP session does not come up

**Check**:

1. **IP connectivity**:
   ```
   ping 10.255.255.2
   ```

2. **TCP port 646**:
   ```
   telnet 10.255.255.2 646
   ```

3. **MPLS on the interface**:
   ```
   show mpls interface
   ```

4. **LDP router-id**:
   ```
   show mpls ldp neighbor
   ```

5. **Firewall**:
   ```
   # Allow LDP
   set firewall ipv4 input filter rule 100 action accept
   set firewall ipv4 input filter rule 100 destination port 646
   set firewall ipv4 input filter rule 100 protocol tcp

   set firewall ipv4 input filter rule 101 action accept
   set firewall ipv4 input filter rule 101 destination port 646
   set firewall ipv4 input filter rule 101 protocol udp

   commit
   ```

6. **IGP reachability**:
   ```
   show ip route 10.255.255.2
   ```

   The transport address must be reachable through the IGP.

### Labels are not distributed

**Check**:

1. **LDP operational**:
   ```
   show mpls ldp neighbor
   ```

   The State should be OPERATIONAL.

2. **IGP routes**:
   ```
   show ip route ospf
   ```

   There must be routes to the prefixes for LDP binding.

3. **LDP bindings**:
   ```
   show mpls ldp binding
   ```

4. **Label range**:
   ```
   show mpls label table
   ```

5. **Debug**:
   ```
   debug mpls ldp messages
   ```

   Verify the exchange of label mapping messages.

### MPLS forwarding does not work

**Check**:

1. **MPLS table**:
   ```
   show mpls table
   ```

   There should be entries for the destination prefix.

2. **LSP path**:
   ```
   traceroute mpls ipv4 192.168.1.1
   ```

3. **TTL propagation**:
   ```
   show configuration | grep ttl-propagation
   ```

4. **Interface MTU**:
   MPLS adds 4 bytes per label, so check the MTU:
   ```
   show interfaces ethernet eth1
   ```

5. **Label operations**:
   - Ingress: Check the push operation
   - Transit: Check the swap operation
   - Egress: Check the pop operation

### High CPU usage

**Causes**:

1. **Too many LDP sessions**
2. **Frequent topology changes**
3. **Debug is enabled**

**Solutions**:

1. **Disable debug**:
   ```
   no debug mpls ldp all
   ```

2. **Tune LDP timers**:
   ```
   set protocols mpls ldp discovery hello-interval 10
   set protocols mpls ldp discovery hello-holdtime 30
   commit
   ```

3. **Reduce LDP sessions** (use targeted LDP only where necessary)

### Label allocation issues

**Check**:

1. **Label range**:
   ```
   show mpls label table
   ```

2. **Available labels**:
   By default: 16-1048575 (1048560 labels)

3. **Label exhaustion**:
   If labels run out (unlikely), increase the range:
   ```
   set protocols mpls label-range dynamic-range-start 16
   set protocols mpls label-range dynamic-range-end 1048575
   commit
   ```

## Best practices

### Network planning

1. **Topology design**:
   - Clear separation of Core/Edge
   - Redundant paths for resilience
   - Hierarchical design (Access-Aggregation-Core)

2. **Addressing**:
   - A dedicated loopback subnet for LSRs
   - A clear numbering scheme
   - Documentation

3. **IGP choice**:
   - OSPF for smaller networks
   - IS-IS for Service Providers
   - Metric optimization for traffic engineering

### Configuration

1. **Always use a loopback**:
   ```
   set protocols mpls ldp router-id 10.255.255.1
   set protocols mpls ldp discovery transport-ipv4-address 10.255.255.1
   ```

2. **MD5 authentication** for critical LDP sessions:
   ```
   set protocols mpls ldp neighbor 10.255.255.2 password 'SecurePassword!'
   ```

3. **TTL security** to protect against spoofing:
   ```
   set protocols mpls ldp neighbor 10.255.255.2 ttl-security hops 1
   ```

4. **Session protection**:
   ```
   set protocols mpls ldp session holdtime 180
   set protocols mpls ldp session keepalive-interval 60
   ```

5. **Graceful restart** (if supported):
   Preserves forwarding while the LDP process restarts.

### Scalability

1. **Label management**:
   - Use liberal retention mode
   - Optimize the label range

2. **LDP optimizations**:
   - Targeted LDP only where necessary
   - Tune timers to balance convergence/overhead

3. **BGP for L3VPN**:
   - Route Reflectors for scaling iBGP
   - Route filtering

### Monitoring

1. **LDP sessions**:
   ```
   show mpls ldp neighbor
   ```
   Monitor the state and uptime.

2. **Label bindings**:
   ```
   show mpls ldp binding summary
   ```

3. **MPLS forwarding**:
   ```
   show mpls table
   ```

4. **Performance metrics**:
   - Label operations per second
   - Control plane CPU usage
   - LDP message rate

5. **Alerting**:
   - LDP neighbor down
   - High label allocation
   - Interface flapping

### Security

1. **Control plane protection**:
   ```
   # Firewall for LDP
   set firewall ipv4 input filter rule 100 action accept
   set firewall ipv4 input filter rule 100 source address 10.255.255.0/24
   set firewall ipv4 input filter rule 100 destination port 646
   set firewall ipv4 input filter rule 100 protocol tcp
   ```

2. **Management plane security**:
   - A separate management VRF
   - SSH keys instead of passwords
   - RBAC for operators

3. **Logging**:
   ```
   set system syslog global facility protocols level info
   ```

### Resilience

1. **Redundant topology**:
   - At least 2 paths between PEs
   - No single point of failure

2. **BFD integration** (when available):
   Fast failure detection.

3. **IGP tuning**:
   ```
   # Faster OSPF convergence
   set protocols ospf timers throttle spf delay 50
   set protocols ospf timers throttle spf initial-holdtime 200
   set protocols ospf timers throttle spf max-holdtime 5000
   ```

4. **Graceful shutdown**:
   Before maintenance:
   ```
   # Drain traffic from LSR
   set protocols ospf max-metric router-lsa administrative
   commit
   ```

## Performance Tuning

### Label Forwarding

**Optimizing the LFIB**:
- Hardware offload (if an ASIC supports it)
- Label stacking depth (memory/functionality trade-off)

### Control Plane

**LDP tuning**:

```
# Adjust hello timers (balance detection time vs overhead)
set protocols mpls ldp discovery hello-interval 5
set protocols mpls ldp discovery hello-holdtime 15

# Session timers
set protocols mpls ldp session holdtime 180
set protocols mpls ldp session keepalive-interval 60

commit
```

**Recommendations**:
- Hello interval: 5-10 seconds (default 5)
- Hold time: at least 3x the hello interval
- Keepalive: 60 seconds (standard)

### IGP Integration

**OSPF optimizations**:

```
# Fast convergence
set protocols ospf timers throttle spf delay 50
set protocols ospf timers throttle spf initial-holdtime 200
set protocols ospls timers throttle spf max-holdtime 5000

# LSA throttling
set protocols ospf timers lsa min-arrival 100

commit
```

**Metric tuning**:
Use OSPF/IS-IS metrics to influence LSP paths.

## Integration with other technologies

### MPLS + BGP

**Use case**: Scalable L3VPN

```
# BGP for customer routes
set protocols bgp system-as 65000
set protocols bgp neighbor 10.255.255.2 remote-as 65000
set protocols bgp neighbor 10.255.255.2 update-source 10.255.255.1

# LDP for label distribution
set protocols mpls ldp router-id 10.255.255.1
set protocols mpls ldp interface eth1

# IGP for reachability
set protocols ospf area 0 network 10.255.255.1/32
```

**Result**: BGP carries customer routes, MPLS provides the forwarding.

### MPLS + QoS

**Traffic Class** in the MPLS header:

```
# QoS policy
set qos policy shaper MPLS-SHAPE class 1 bandwidth 100mbit
set qos policy shaper MPLS-SHAPE class 1 match VOICE dscp ef

set interfaces ethernet eth1 traffic-policy out MPLS-SHAPE

commit
```

**EXP bits** are copied from the IP DSCP (if TTL propagation is enabled).

### MPLS + IPsec

**Encrypting MPLS traffic** (for sensitive data):

```
# IPsec tunnel for MPLS
set vpn ipsec esp-group MPLS-ESP mode tunnel
set vpn ipsec esp-group MPLS-ESP pfs enable
set vpn ipsec esp-group MPLS-ESP proposal 1 encryption aes256
set vpn ipsec esp-group MPLS-ESP proposal 1 hash sha256

set vpn ipsec ike-group MPLS-IKE proposal 1 encryption aes256
set vpn ipsec ike-group MPLS-IKE proposal 1 hash sha256

set vpn ipsec site-to-site peer 203.0.113.1 ike-group MPLS-IKE
set vpn ipsec site-to-site peer 203.0.113.1 tunnel 1 esp-group MPLS-ESP
set vpn ipsec site-to-site peer 203.0.113.1 tunnel 1 local prefix 10.0.0.0/8
set vpn ipsec site-to-site peer 203.0.113.1 tunnel 1 remote prefix 10.0.0.0/8

commit
```

**Note**: MPLS over IPsec adds overhead but provides confidentiality.

## Migrating to MPLS

### Rollout phases

**Phase 1: Planning**
1. Audit the existing network
2. Define requirements (bandwidth, QoS, VPN)
3. Choose a design (LDP vs RSVP-TE)
4. Lab testing

**Phase 2: Core Deployment**
1. Deploy MPLS on core routers (P)
2. Configure the IGP
3. Enable LDP
4. Verify LSPs

**Phase 3: Edge Deployment**
1. Configure PE routers
2. Configure VRFs (for L3VPN)
3. BGP integration
4. Test with pilot customers

**Phase 4: Migration**
1. Gradually move traffic onto MPLS
2. Monitor performance
3. Optimization
4. Decommission legacy networks

### Parallel Running

**Running MPLS in parallel with IP**:

```
# Existing IP routing (OSPF)
set protocols ospf area 0 network 10.0.0.0/8

# New MPLS/LDP
set protocols mpls interface eth1
set protocols mpls ldp interface eth1
set protocols mpls ldp router-id 10.255.255.1

commit
```

**Advantages**:
- Zero downtime migration
- Rollback capability
- Gradual testing

**Disadvantages**:
- Increased complexity
- Additional resources

## Next steps

- [BGP](/docs/vyos/routing/vyos-bgp/) - for L3VPN and inter-AS routing
- [OSPF](/docs/vyos/routing/vyos-ospf/) - IGP for the MPLS core
- [VRF](/docs/vyos/vrf/) - Virtual Routing and Forwarding
- [QoS](/docs/vyos/qos/) - Quality of Service with MPLS
- [Firewall](/docs/vyos/firewall/) - protecting the MPLS network

## Reference materials

**RFC**:
- RFC 3031 - MPLS Architecture
- RFC 3032 - MPLS Label Stack Encoding
- RFC 5036 - LDP Specification
- RFC 4364 - BGP/MPLS IP VPNs (L3VPN)
- RFC 3209 - RSVP-TE Extensions for LSP Tunnels
- RFC 4379 - LSP Ping and Traceroute
- RFC 5920 - Security Framework for MPLS

**VyOS Documentation**:
- https://docs.vyos.io/en/latest/configuration/protocols/mpls.html

**Books**:
- "MPLS Fundamentals" - Luc De Ghein
- "MPLS and VPN Architectures" - Pepelnjak, Guichard
- "Traffic Engineering with MPLS" - Osborne, Simha