# Segment Routing in VyOS

> Segment Routing (SR-MPLS and SRv6) in VyOS - configuration with IS-IS and OSPF, SR policies, TI-LFA, and examples for cloud environments

Source: https://opennix.org/en/docs/vyos/routing/vyos-segment-routing/


## Segment Routing

## Overview

Segment Routing (SR) is a routing architecture based on source routing that lets the sending node determine the path packets take through the network. SR simplifies traffic management and path optimization without requiring state to be maintained on intermediate routers.

### Key characteristics

- **Simplified architecture**: No signaling protocol required (LDP, RSVP-TE)
- **Scalability**: Minimal state on intermediate nodes
- **Traffic Engineering**: Flexible control of traffic flows
- **Fast convergence**: Support for TI-LFA for instant switchover to a backup path
- **Compatibility**: Works on top of existing MPLS infrastructure

### Types of Segment Routing

VyOS supports two types of Segment Routing:

1. **SR-MPLS**: Uses the existing MPLS data plane
2. **SRv6**: Uses IPv6 with extension headers (under development)

This guide focuses on SR-MPLS, which is a production-ready solution.

## Segment Routing architecture

### Core components

#### Segment

A segment represents an instruction that a node must perform on a packet. Segments are identified by a **Segment Identifier (SID)**.

#### Segment Identifier (SID)

A SID is a unique identifier of a segment. In SR-MPLS it is implemented as an MPLS label.

**SID types**:

- **Prefix-SID**: A global segment associated with an IGP prefix
- **Adjacency-SID**: A local segment representing a specific adjacency
- **Node-SID**: A special case of Prefix-SID for a node's loopback interface
- **Anycast-SID**: A SID assigned to multiple nodes for load balancing

#### Segment Routing Global Block (SRGB)

The SRGB is a global range of MPLS labels used for Prefix-SIDs. All nodes in an SR domain must use the same SRGB to ensure correct operation.

**SRGB parameters**:
- **Lower Bound**: Starting value of the range (default: 16000)
- **Upper Bound**: Ending value of the range (default: 23999)
- **Range Size**: Size of the range (maximum: 65535)

#### Segment Routing Local Block (SRLB)

The SRLB is a local range of labels used for Adjacency-SIDs and other local segments.

### How SR-MPLS works

1. **Initialization**: The ingress router determines the path and builds the MPLS label stack
2. **Forwarding**: Intermediate routers process the top label in the stack
3. **Termination**: The egress router removes the last label and delivers the packet

**Label operations**:

- **PUSH**: Add a label to the stack (on the ingress node)
- **SWAP**: Replace the top label (on transit nodes)
- **POP**: Remove the top label
- **CONTINUE**: Move to the next label in the stack

### PHP (Penultimate Hop Popping)

PHP is a mechanism in which the penultimate router removes the top label before forwarding the packet to the egress node. This reduces the load on the egress router.

**Modes**:

- **Implicit-null**: Label 3 (default with PHP)
- **Explicit-null**: Label 0 for IPv4, label 2 for IPv6 (with the explicit-null flag)
- **No-PHP**: Disables PHP (with the no-php-flag)

## SR-MPLS configuration

### System requirements

- VyOS 1.4 (Sagitta) or newer
- MPLS support in the Linux kernel
- A configured IGP (IS-IS or OSPF)

### Basic SRGB configuration

```bash
set protocols segment-routing global-block high-label-value 23999
set protocols segment-routing global-block low-label-value 16000
```

**Parameters**:

- `high-label-value`: Upper bound of the SRGB (maximum: 1048575)
- `low-label-value`: Lower bound of the SRGB (minimum: 0)

**Important**:
- The SRGB size must not exceed 65535 labels
- The SRGB must be identical on all nodes in the SR domain
- It is recommended to use a range that does not conflict with dynamic LDP labels

### Maximum Label Stack Depth

```bash
set protocols segment-routing maximum-label-stack-depth 10
```

Defines the maximum number of labels in the stack (range: 1-16). Default: 10.

## Segment Routing with IS-IS

### Enabling SR in IS-IS

```bash
set protocols isis SR segment-routing enable
```

### SRGB configuration for IS-IS

```bash
set protocols isis SR segment-routing global-block high-label-value 23999
set protocols isis SR segment-routing global-block low-label-value 16000
```

### Configuring the Prefix-SID

A Prefix-SID is assigned to the loopback interface to create a Node-SID:

```bash
set protocols isis SR segment-routing prefix 10.0.0.1/32 index value 1
set protocols isis SR segment-routing prefix 10.0.0.1/32 index no-php-flag
```

**Prefix-SID parameters**:

- `value`: Index within the SRGB (0-65535)
- `no-php-flag`: Disables PHP for this prefix
- `explicit-null`: Uses an explicit-null label instead of implicit-null
- `n-flag-clear`: Clears the N-flag in the TLV (for special cases)

**Label calculation**: `Label = SRGB_Lower_Bound + Index`

Example: If SRGB = 16000-23999 and Index = 1, then the label = 16001

### Complete IS-IS SR configuration example

**Node R1 (10.0.0.1)**:

```bash
# Basic IS-IS configuration
set protocols isis interface eth1
set protocols isis interface lo
set protocols isis net 49.0000.0000.0001.00
set protocols isis level level-2

# Segment Routing
set protocols isis SR segment-routing enable
set protocols isis SR segment-routing global-block low-label-value 16000
set protocols isis SR segment-routing global-block high-label-value 23999
set protocols isis SR segment-routing prefix 10.0.0.1/32 index value 1

# MPLS on interfaces
set interfaces ethernet eth1 ip address 192.168.1.1/24
set interfaces loopback lo address 10.0.0.1/32
set protocols mpls interface eth1
```

**Node R2 (10.0.0.2)**:

```bash
# Basic IS-IS configuration
set protocols isis interface eth1
set protocols isis interface eth2
set protocols isis interface lo
set protocols isis net 49.0000.0000.0002.00
set protocols isis level level-2

# Segment Routing
set protocols isis SR segment-routing enable
set protocols isis SR segment-routing global-block low-label-value 16000
set protocols isis SR segment-routing global-block high-label-value 23999
set protocols isis SR segment-routing prefix 10.0.0.2/32 index value 2

# MPLS on interfaces
set interfaces ethernet eth1 ip address 192.168.1.2/24
set interfaces ethernet eth2 ip address 192.168.2.1/24
set interfaces loopback lo address 10.0.0.2/32
set protocols mpls interface eth1
set protocols mpls interface eth2
```

**Node R3 (10.0.0.3)**:

```bash
# Basic IS-IS configuration
set protocols isis interface eth1
set protocols isis interface lo
set protocols isis net 49.0000.0000.0003.00
set protocols isis level level-2

# Segment Routing
set protocols isis SR segment-routing enable
set protocols isis SR segment-routing global-block low-label-value 16000
set protocols isis SR segment-routing global-block high-label-value 23999
set protocols isis SR segment-routing prefix 10.0.0.3/32 index value 3

# MPLS on interfaces
set interfaces ethernet eth1 ip address 192.168.2.2/24
set interfaces loopback lo address 10.0.0.3/32
set protocols mpls interface eth1
```

## Segment Routing with OSPF

### Enabling SR in OSPF

```bash
set protocols ospf segment-routing enable
```

### SRGB configuration for OSPF

```bash
set protocols ospf segment-routing global-block high-label-value 23999
set protocols ospf segment-routing global-block low-label-value 16000
```

### Configuring the Prefix-SID for OSPF

```bash
set protocols ospf segment-routing prefix 10.0.0.1/32 index value 1
set protocols ospf segment-routing prefix 10.0.0.1/32 index no-php-flag
```

### Complete OSPF SR configuration example

**Node R1 (10.0.0.1)**:

```bash
# Basic OSPF configuration
set interfaces ethernet eth1 ip address 192.168.1.1/24
set interfaces loopback lo address 10.0.0.1/32

set protocols ospf area 0 network 192.168.1.0/24
set protocols ospf area 0 network 10.0.0.1/32
set protocols ospf parameters router-id 10.0.0.1

# Segment Routing
set protocols ospf segment-routing enable
set protocols ospf segment-routing global-block low-label-value 16000
set protocols ospf segment-routing global-block high-label-value 23999
set protocols ospf segment-routing prefix 10.0.0.1/32 index value 1

# MPLS
set protocols mpls interface eth1
```

**Node R2 (10.0.0.2)**:

```bash
# Basic OSPF configuration
set interfaces ethernet eth1 ip address 192.168.1.2/24
set interfaces ethernet eth2 ip address 192.168.2.1/24
set interfaces loopback lo address 10.0.0.2/32

set protocols ospf area 0 network 192.168.1.0/24
set protocols ospf area 0 network 192.168.2.0/24
set protocols ospf area 0 network 10.0.0.2/32
set protocols ospf parameters router-id 10.0.0.2

# Segment Routing
set protocols ospf segment-routing enable
set protocols ospf segment-routing global-block low-label-value 16000
set protocols ospf segment-routing global-block high-label-value 23999
set protocols ospf segment-routing prefix 10.0.0.2/32 index value 2

# MPLS
set protocols mpls interface eth1
set protocols mpls interface eth2
```

**Node R3 (10.0.0.3)**:

```bash
# Basic OSPF configuration
set interfaces ethernet eth1 ip address 192.168.2.2/24
set interfaces loopback lo address 10.0.0.3/32

set protocols ospf area 0 network 192.168.2.0/24
set protocols ospf area 0 network 10.0.0.3/32
set protocols ospf parameters router-id 10.0.0.3

# Segment Routing
set protocols ospf segment-routing enable
set protocols ospf segment-routing global-block low-label-value 16000
set protocols ospf segment-routing global-block high-label-value 23999
set protocols ospf segment-routing prefix 10.0.0.3/32 index value 3

# MPLS
set protocols mpls interface eth1
```

## SR Policies and explicit paths

SR Policies let you define explicit paths for traffic using a list of SIDs.

### The SR Policy concept

An SR Policy consists of:

- **Endpoint**: The target node (IP address)
- **Color**: A policy identifier used to group paths
- **Candidate Path**: A list of possible paths
- **Segment List**: An ordered list of SIDs that define the path

### SR Policy use cases

**Scenario 1: Low-latency path**

For latency-sensitive applications, you can configure a path over high-speed links:

```bash
# Marking VoIP traffic to use the low-latency path
set policy route LowLatency rule 10 protocol udp
set policy route LowLatency rule 10 destination port 5060
set policy route LowLatency rule 10 set table 100

# Static route using SR
set protocols static route 10.0.0.10/32 interface eth1 segments 16005 16008 16010
```

**Scenario 2: Disjoint paths for resiliency**

Creating two non-overlapping paths for critical traffic:

```bash
# Primary path: R1 -> R2 -> R4 -> R5
# Backup path: R1 -> R3 -> R6 -> R5

# Primary SR policy (color 10)
set policy route PrimaryPath rule 10 destination address 10.10.0.0/16
set policy route PrimaryPath rule 10 set table 10

# Backup SR policy (color 20) - activated when the primary fails
set policy route BackupPath rule 10 destination address 10.10.0.0/16
set policy route BackupPath rule 10 set table 20
```

## TI-LFA (Topology Independent Loop-Free Alternate)

TI-LFA uses Segment Routing to create loop-free backup paths that are independent of the network topology.

### Benefits of TI-LFA

- **Instant convergence**: Switchover to a backup path in <50ms
- **Topology independence**: Works in any topology, including ring and mesh
- **Loop-free**: Guaranteed absence of routing loops
- **Optimal paths**: The backup path is close to optimal

### Enabling TI-LFA in IS-IS

```bash
set protocols isis fast-reroute lfa local load-sharing disable
set protocols isis fast-reroute lfa local priority-limit critical
set protocols isis interface eth1 fast-reroute lfa local enable
set protocols isis interface eth1 fast-reroute lfa local tiebreaker downstream index 10
```

**Parameters**:

- `local enable`: Enables LFA on the interface
- `load-sharing`: Balancing across multiple LFA paths
- `priority-limit`: Protects only critical prefixes
- `tiebreaker`: Criteria for selecting the LFA when several options exist

### Enabling TI-LFA in OSPF

```bash
set protocols ospf fast-reroute ti-lfa enable
set protocols ospf fast-reroute ti-lfa load-sharing disable
set protocols ospf interface eth1 fast-reroute ti-lfa enable
```

### TI-LFA configuration example

**Topology**: R1 --- R2 --- R4
                  \   /
                   R3

```bash
# R1 Configuration
set protocols isis interface eth1 fast-reroute lfa local enable
set protocols isis interface eth2 fast-reroute lfa local enable
set protocols isis fast-reroute lfa local priority-limit critical

# R2 Configuration
set protocols isis interface eth1 fast-reroute lfa local enable
set protocols isis interface eth2 fast-reroute lfa local enable
set protocols isis interface eth3 fast-reroute lfa local enable

# R3 Configuration
set protocols isis interface eth1 fast-reroute lfa local enable
set protocols isis interface eth2 fast-reroute lfa local enable
```

If the R1-R2 link fails, traffic instantly switches over via R3 using a pre-computed SR path.

## Examples for cloud providers

### Example 1: Yandex Cloud - SR for Traffic Engineering in a data center

**Scenario**: An operator wants to manage traffic between availability zones in Yandex Cloud using SR to optimize bandwidth.

**Topology**:
- DC1 (ru-central1-a): R1, R2
- DC2 (ru-central1-b): R3, R4
- Core: R5, R6 (backbone between data centers)

**Requirements**:
- Critical database traffic goes over a dedicated high-bandwidth path
- Web traffic uses the shortest path
- Redundancy via TI-LFA

**R1 configuration (DC1)**:

```bash
# Basic interfaces
set interfaces ethernet eth0 address 10.128.0.10/24
set interfaces ethernet eth1 address 172.16.1.1/30
set interfaces ethernet eth2 address 172.16.2.1/30
set interfaces loopback lo address 10.0.1.1/32

# IS-IS + SR
set protocols isis interface eth0
set protocols isis interface eth1
set protocols isis interface eth2
set protocols isis interface lo
set protocols isis net 49.0001.0000.0001.00
set protocols isis level level-2

set protocols isis SR segment-routing enable
set protocols isis SR segment-routing global-block low-label-value 20000
set protocols isis SR segment-routing global-block high-label-value 29999
set protocols isis SR segment-routing prefix 10.0.1.1/32 index value 1

# MPLS
set protocols mpls interface eth1
set protocols mpls interface eth2

# TI-LFA for fast convergence
set protocols isis interface eth1 fast-reroute lfa local enable
set protocols isis interface eth2 fast-reroute lfa local enable

# Traffic Engineering for the database
set policy route DB_TRAFFIC rule 10 source address 10.128.0.0/16
set policy route DB_TRAFFIC rule 10 destination port 5432
set policy route DB_TRAFFIC rule 10 protocol tcp
set policy route DB_TRAFFIC rule 10 set table 100

# Explicit path for the database: R1 -> R5 -> R4 (via the high-bandwidth core)
set protocols static table 100 route 10.129.0.0/16 segments 20005 20004
```

**R5 configuration (Core Backbone)**:

```bash
# Interfaces
set interfaces ethernet eth0 address 172.16.1.2/30
set interfaces ethernet eth1 address 172.16.3.1/30
set interfaces ethernet eth2 address 172.16.5.1/30
set interfaces loopback lo address 10.0.5.5/32

# IS-IS + SR
set protocols isis interface eth0
set protocols isis interface eth1
set protocols isis interface eth2
set protocols isis interface lo
set protocols isis net 49.0001.0000.0005.00
set protocols isis level level-2

set protocols isis SR segment-routing enable
set protocols isis SR segment-routing global-block low-label-value 20000
set protocols isis SR segment-routing global-block high-label-value 29999
set protocols isis SR segment-routing prefix 10.0.5.5/32 index value 5

# MPLS on all interfaces
set protocols mpls interface eth0
set protocols mpls interface eth1
set protocols mpls interface eth2

# TI-LFA
set protocols isis interface eth0 fast-reroute lfa local enable
set protocols isis interface eth1 fast-reroute lfa local enable
set protocols isis interface eth2 fast-reroute lfa local enable

# QoS for prioritizing DB traffic
set traffic-policy shaper DB_SHAPER bandwidth 1gbit
set traffic-policy shaper DB_SHAPER default bandwidth 900mbit
set traffic-policy shaper DB_SHAPER class 10 bandwidth 100mbit
set traffic-policy shaper DB_SHAPER class 10 match DB_MATCH ip dscp ef
set interfaces ethernet eth1 traffic-policy out DB_SHAPER
```

**R4 configuration (DC2)**:

```bash
# Interfaces
set interfaces ethernet eth0 address 10.129.0.10/24
set interfaces ethernet eth1 address 172.16.3.2/30
set interfaces ethernet eth2 address 172.16.4.2/30
set interfaces loopback lo address 10.0.4.4/32

# IS-IS + SR
set protocols isis interface eth0
set protocols isis interface eth1
set protocols isis interface eth2
set protocols isis interface lo
set protocols isis net 49.0001.0000.0004.00
set protocols isis level level-2

set protocols isis SR segment-routing enable
set protocols isis SR segment-routing global-block low-label-value 20000
set protocols isis SR segment-routing global-block high-label-value 29999
set protocols isis SR segment-routing prefix 10.0.4.4/32 index value 4

# MPLS
set protocols mpls interface eth1
set protocols mpls interface eth2

# TI-LFA
set protocols isis interface eth1 fast-reroute lfa local enable
set protocols isis interface eth2 fast-reroute lfa local enable
```

**Result**:
- PostgreSQL traffic (port 5432) from DC1 to DC2 goes over the dedicated path R1->R5->R4
- All other traffic uses the shortest path (which may go via R6)
- If any link fails, TI-LFA provides instant switchover

### Example 2: VK Cloud - SR-MPLS backbone

**Scenario**: Building an MPLS backbone for VK Cloud using SR instead of LDP to simplify management.

**Topology**:
- Region Moscow: PE1, PE2
- Region SPb: PE3, PE4
- Core: P1, P2, P3 (mesh topology)

**Highlights**:
- Anycast SID for load balancing to a group of PEs
- SR Policies for different traffic classes
- OSPF as the IGP
- TI-LFA for sub-50ms failover

**PE1 configuration (Moscow)**:

```bash
# Interfaces
set interfaces ethernet eth0 address 10.0.0.1/32
set interfaces ethernet eth1 address 192.168.1.1/30
set interfaces ethernet eth2 address 192.168.2.1/30
set interfaces loopback lo address 10.0.0.1/32

# OSPF + SR
set protocols ospf area 0 network 192.168.1.0/30
set protocols ospf area 0 network 192.168.2.0/30
set protocols ospf area 0 network 10.0.0.1/32
set protocols ospf parameters router-id 10.0.0.1

set protocols ospf segment-routing enable
set protocols ospf segment-routing global-block low-label-value 16000
set protocols ospf segment-routing global-block high-label-value 23999
set protocols ospf segment-routing prefix 10.0.0.1/32 index value 1

# Anycast SID for the Moscow PE group
set protocols ospf segment-routing prefix 10.100.1.0/32 index value 101

# MPLS
set protocols mpls interface eth1
set protocols mpls interface eth2

# TI-LFA
set protocols ospf fast-reroute ti-lfa enable
set protocols ospf interface eth1 fast-reroute ti-lfa enable
set protocols ospf interface eth2 fast-reroute ti-lfa enable

# L3VPN for customers
set protocols bgp system-as 65000
set protocols bgp neighbor 10.0.0.3 remote-as 65000
set protocols bgp neighbor 10.0.0.3 update-source lo
set protocols bgp neighbor 10.0.0.3 address-family ipv4-vpn

set vrf name CUSTOMER_A table 1000
set vrf name CUSTOMER_A protocols bgp system-as 65000
set vrf name CUSTOMER_A protocols bgp neighbor 172.16.1.2 remote-as 65001
```

**P1 configuration (Core)**:

```bash
# Interfaces (transit only)
set interfaces ethernet eth0 address 192.168.1.2/30
set interfaces ethernet eth1 address 192.168.10.1/30
set interfaces ethernet eth2 address 192.168.11.1/30
set interfaces ethernet eth3 address 192.168.12.1/30
set interfaces loopback lo address 10.0.10.1/32

# OSPF + SR
set protocols ospf area 0 network 192.168.1.0/30
set protocols ospf area 0 network 192.168.10.0/30
set protocols ospf area 0 network 192.168.11.0/30
set protocols ospf area 0 network 192.168.12.0/30
set protocols ospf area 0 network 10.0.10.1/32
set protocols ospf parameters router-id 10.0.10.1

set protocols ospf segment-routing enable
set protocols ospf segment-routing global-block low-label-value 16000
set protocols ospf segment-routing global-block high-label-value 23999
set protocols ospf segment-routing prefix 10.0.10.1/32 index value 10

# MPLS on all interfaces
set protocols mpls interface eth0
set protocols mpls interface eth1
set protocols mpls interface eth2
set protocols mpls interface eth3

# TI-LFA on all interfaces
set protocols ospf fast-reroute ti-lfa enable
set protocols ospf interface eth0 fast-reroute ti-lfa enable
set protocols ospf interface eth1 fast-reroute ti-lfa enable
set protocols ospf interface eth2 fast-reroute ti-lfa enable
set protocols ospf interface eth3 fast-reroute ti-lfa enable

# MPLS TTL propagation
set protocols mpls no-propagate-ttl
```

**PE3 configuration (SPb)**:

```bash
# Interfaces
set interfaces ethernet eth0 address 10.0.0.3/32
set interfaces ethernet eth1 address 192.168.20.1/30
set interfaces ethernet eth2 address 192.168.21.1/30
set interfaces loopback lo address 10.0.0.3/32

# OSPF + SR
set protocols ospf area 0 network 192.168.20.0/30
set protocols ospf area 0 network 192.168.21.0/30
set protocols ospf area 0 network 10.0.0.3/32
set protocols ospf parameters router-id 10.0.0.3

set protocols ospf segment-routing enable
set protocols ospf segment-routing global-block low-label-value 16000
set protocols ospf segment-routing global-block high-label-value 23999
set protocols ospf segment-routing prefix 10.0.0.3/32 index value 3

# Anycast SID for the SPb PE group
set protocols ospf segment-routing prefix 10.100.2.0/32 index value 102

# MPLS
set protocols mpls interface eth1
set protocols mpls interface eth2

# TI-LFA
set protocols ospf fast-reroute ti-lfa enable
set protocols ospf interface eth1 fast-reroute ti-lfa enable
set protocols ospf interface eth2 fast-reroute ti-lfa enable

# L3VPN for customers
set protocols bgp system-as 65000
set protocols bgp neighbor 10.0.0.1 remote-as 65000
set protocols bgp neighbor 10.0.0.1 update-source lo
set protocols bgp neighbor 10.0.0.1 address-family ipv4-vpn

set vrf name CUSTOMER_A table 1000
set vrf name CUSTOMER_A protocols bgp system-as 65000
set vrf name CUSTOMER_A protocols bgp neighbor 172.16.2.2 remote-as 65001
```

**SR Policies for different traffic classes**:

On PE1, configure different paths for different types of traffic:

```bash
# Real-time traffic (VoIP, video) - shortest path
set policy route REALTIME rule 10 protocol udp
set policy route REALTIME rule 10 destination port 5060-5080
set policy route REALTIME rule 10 set dscp ef

# Bulk data - over a less-loaded path
set policy route BULK rule 10 protocol tcp
set policy route BULK rule 10 destination port 873
set policy route BULK rule 10 set table 200

# Static route with SR for bulk: PE1 -> P2 -> P3 -> PE3
set protocols static table 200 route 10.0.0.3/32 segments 16012 16013 16003
```

**Result**:
- Simplified configuration without LDP
- Anycast SID allows traffic to be balanced between PEs in the same region
- TI-LFA provides fast convergence
- SR Policies optimize bandwidth usage

## Verification and monitoring commands

### Verifying the SR configuration

```bash
# Show the SRGB configuration
show running-config protocols segment-routing

# Output:
# protocols {
#     segment-routing {
#         global-block {
#             high-label-value 23999
#             low-label-value 16000
#         }
#         maximum-label-stack-depth 10
#     }
# }
```

### Verifying SR in IS-IS

```bash
# General SR information in IS-IS
show isis segment-routing

# Node-SID information
show isis segment-routing node

# Prefixes with SIDs
show isis segment-routing prefix-sids
```

**Example output**:

```
 Area  Prefix                 SID Type       Flags  Algorithm
 ------------------------------------------------------------------
 Level-2 10.0.0.1/32          1   Node       R:0 N:1 P:0 E:0 V:0 L:0  SPF
 Level-2 10.0.0.2/32          2   Node       R:0 N:1 P:0 E:0 V:0 L:0  SPF
 Level-2 10.0.0.3/32          3   Node       R:0 N:1 P:0 E:0 V:0 L:0  SPF
```

**Flags**:
- **R**: Re-advertisement flag
- **N**: Node-SID flag (this SID represents a node)
- **P**: no-PHP flag
- **E**: Explicit-null flag
- **V**: Value flag (the SID is an absolute label rather than an index)
- **L**: Local flag

### Verifying SR in OSPF

```bash
# General SR information in OSPF
show ip ospf segment-routing

# Per-prefix details
show ip ospf segment-routing prefix-sids
```

**Example output**:

```
OSPF Segment Routing:
  SR Enabled: True
  SRGB Range: [16000:23999]
  Algorithm: SPF
  MSD: 10

Prefix-SIDs:
  10.0.0.1/32 - Index 1 (Label 16001) Flags: N
  10.0.0.2/32 - Index 2 (Label 16002) Flags: N
  10.0.0.3/32 - Index 3 (Label 16003) Flags: N
```

### Verifying the MPLS table

```bash
# Show local MPLS labels
show mpls table

# Show the MPLS forwarding table
show mpls fib
```

**Example output of show mpls table**:

```
 Inbound Label  Type       Nexthop               Outbound Label
 ----------------------------------------------------------------
 16001          SR (IS-IS) local                 -
 16002          SR (IS-IS) 192.168.1.2 via eth1  implicit-null
 16003          SR (IS-IS) 192.168.1.2 via eth1  16003
 16003          SR (IS-IS) 192.168.2.2 via eth2  16003
```

### Verifying SR paths

```bash
# Traceroute showing MPLS labels
traceroute mpls 10.0.0.3

# IP traceroute
traceroute 10.0.0.3
```

**Example output**:

```
traceroute to 10.0.0.3 (10.0.0.3), 30 hops max, 60 byte packets
 1  192.168.1.2 (192.168.1.2) [MPLS: Label 16003]  0.521 ms  0.489 ms  0.467 ms
 2  192.168.2.2 (192.168.2.2) [MPLS: Label-switched]  0.891 ms  0.854 ms  0.812 ms
 3  10.0.0.3 (10.0.0.3)  1.123 ms  1.089 ms  1.056 ms
```

### Verifying TI-LFA

```bash
# Show computed LFA for IS-IS
show isis fast-reroute summary

# Detailed LFA information
show isis fast-reroute detail
```

**Example output**:

```
IS-IS Fast-Reroute Summary:
  Enabled: True
  Type: TI-LFA

Prefix             Primary-NH      Backup-NH       Backup-Label
------------------------------------------------------------------
10.0.0.3/32        192.168.1.2     192.168.2.2     16002,16003
10.0.0.4/32        192.168.1.2     192.168.2.2     16002,16004
```

### Verifying the IS-IS/OSPF database

```bash
# IS-IS LSDB with SR information
show isis database detail

# OSPF LSDB with SR extensions
show ip ospf database opaque-area
```

### Monitoring SR performance

```bash
# MPLS statistics
show mpls statistics

# Interface statistics with MPLS
show interfaces ethernet eth1 mpls
```

### Verifying BGP with SR (for L3VPN)

```bash
# BGP routes with MPLS labels
show bgp ipv4 vpn summary

# Detailed information on VPNv4 routes
show bgp ipv4 vpn
```

## Troubleshooting

### Problem 1: SR is not working in IS-IS/OSPF

**Symptoms**:
- The `show isis segment-routing` command does not show prefixes with SIDs
- The MPLS table is empty
- No SR labels appear in traceroute

**Diagnostics**:

```bash
# Check whether SR is enabled
show running-config protocols isis | match segment-routing
show running-config protocols ospf | match segment-routing

# Check the SRGB
show running-config protocols segment-routing

# Check whether Prefix-SIDs are configured
show running-config protocols isis | match "prefix.*index"
show running-config protocols ospf | match "prefix.*index"

# Check MPLS on interfaces
show mpls interface

# Check IGP adjacencies
show isis neighbors
show ip ospf neighbor
```

**Solutions**:

1. **SR is not enabled**:
   ```bash
   set protocols isis SR segment-routing enable
   # or
   set protocols ospf segment-routing enable
   commit
   ```

2. **The SRGB is not configured or is incorrect**:
   ```bash
   set protocols segment-routing global-block low-label-value 16000
   set protocols segment-routing global-block high-label-value 23999
   commit
   ```

3. **No Prefix-SID is assigned**:
   ```bash
   set protocols isis SR segment-routing prefix 10.0.0.1/32 index value 1
   commit
   ```

4. **MPLS is not enabled on interfaces**:
   ```bash
   set protocols mpls interface eth1
   commit
   ```

5. **Different SRGB on nodes** (critical error):
   ```bash
   # Check on all nodes
   show running-config protocols segment-routing global-block
   # Align to a single value
   ```

### Problem 2: TI-LFA does not protect routes

**Symptoms**:
- `show isis fast-reroute summary` shows "No backup available"
- A link failure results in slow convergence
- Packets are lost during failover

**Diagnostics**:

```bash
# Check whether TI-LFA is enabled
show running-config protocols isis | match fast-reroute
show running-config protocols ospf | match ti-lfa

# Check the topology
show isis topology
show ip ospf database

# Check computed paths
show isis fast-reroute detail
```

**Solutions**:

1. **TI-LFA is not enabled globally**:
   ```bash
   set protocols isis fast-reroute lfa local priority-limit critical
   # or for OSPF
   set protocols ospf fast-reroute ti-lfa enable
   commit
   ```

2. **TI-LFA is not enabled on interfaces**:
   ```bash
   set protocols isis interface eth1 fast-reroute lfa local enable
   # or for OSPF
   set protocols ospf interface eth1 fast-reroute ti-lfa enable
   commit
   ```

3. **Insufficient topology connectivity**:
   - TI-LFA requires alternative paths
   - Verify that backup links exist
   - In a linear topology (A-B-C), TI-LFA cannot protect all prefixes

4. **Metrics do not allow finding an LFA**:
   ```bash
   # Check interface metrics
   show isis interface detail
   # Adjust metrics for better balancing
   set protocols isis interface eth2 metric 20
   commit
   ```

### Problem 3: MPLS labels are not assigned correctly

**Symptoms**:
- Labels fall outside the SRGB
- Label conflicts
- `show mpls table` shows unexpected values

**Diagnostics**:

```bash
# Check assigned indexes
show isis segment-routing prefix-sids
show ip ospf segment-routing prefix-sids

# Check the SRGB
show running-config protocols segment-routing global-block

# Check the MPLS table
show mpls table

# Check dynamic LDP labels (if used)
show mpls ldp binding
```

**Solutions**:

1. **The index falls outside the SRGB**:
   ```bash
   # If SRGB = 16000-23999 (size 8000), the index cannot be > 7999
   # Fix the index
   delete protocols isis SR segment-routing prefix 10.0.0.1/32 index value 10000
   set protocols isis SR segment-routing prefix 10.0.0.1/32 index value 1
   commit
   ```

2. **Conflict with dynamic labels**:
   ```bash
   # Check the ranges
   show mpls table
   # Make sure the SRGB does not overlap with LDP labels (typically 1024+)
   # Recommended SRGB: 16000-23999
   ```

3. **Duplicate indexes**:
   ```bash
   # Different nodes must not have the same index for different prefixes
   # Verify uniqueness across all nodes
   show isis segment-routing prefix-sids
   ```

### Problem 4: SR Policies are not working

**Symptoms**:
- Traffic does not follow the explicitly defined path
- Packets take the shortest path instead of the SR policy
- Errors when installing the SR policy

**Diagnostics**:

```bash
# Check routing tables
show ip route table all

# Check policy routing
show policy route

# Traceroute with MPLS
traceroute mpls <destination>

# Check MPLS forwarding
show mpls fib
```

**Solutions**:

1. **Policy routing is not applied on the interface**:
   ```bash
   set interfaces ethernet eth0 policy route MY_POLICY
   commit
   ```

2. **Incorrect SID list**:
   ```bash
   # Make sure all SIDs in the path are reachable
   show isis segment-routing prefix-sids
   # Verify that the path actually exists
   traceroute 10.0.0.5
   ```

3. **A static route with segments is not installed**:
   ```bash
   # Check whether the route exists
   show ip route table 100
   # Install it if it is missing
   set protocols static table 100 route 10.0.0.5/32 segments 16002 16005
   commit
   ```

### Problem 5: High latency or packet loss with SR

**Symptoms**:
- Increased latency on SR paths
- Packet loss when using SR
- MTU issues

**Diagnostics**:

```bash
# Check the MTU on interfaces
show interfaces ethernet eth1

# Check the label stack depth
show running-config protocols segment-routing maximum-label-stack-depth

# MTU testing
ping 10.0.0.3 size 1500 do-not-fragment

# Check MPLS statistics
show mpls statistics
```

**Solutions**:

1. **The MTU does not account for MPLS overhead**:
   ```bash
   # Each MPLS label adds 4 bytes
   # A stack of 3 labels requires +12 bytes
   set interfaces ethernet eth1 mtu 1512
   commit
   ```

2. **The label stack is too large**:
   ```bash
   # Reduce the number of SIDs in the policy
   # Or use a Binding SID (once it is supported)
   set protocols segment-routing maximum-label-stack-depth 5
   commit
   ```

3. **Fragmentation on intermediate nodes**:
   ```bash
   # Enable MPLS MTU on all transit nodes
   set protocols mpls mtu 1500
   commit
   ```

### Problem 6: SR stops working after an FRR upgrade

**Symptoms**:
- After a VyOS/FRR upgrade, SR stopped working
- The configuration is in place, but the functionality does not work

**Diagnostics**:

```bash
# Check the FRR version
show version

# Check FRR logs
show log
run show log | match -i "segment"

# Check the state of the IS-IS/OSPF daemon
show log | match -i isis
show log | match -i ospf
```

**Solutions**:

1. **Restart FRR**:
   ```bash
   restart frr
   ```

2. **Reapply the configuration**:
   ```bash
   # Save the configuration
   save /tmp/config.boot
   # Reboot
   reboot
   # Or
   load /tmp/config.boot
   commit
   ```

3. **Check configuration compatibility**:
   ```bash
   # Some options may have changed
   # Check the VyOS changelog
   # Update the syntax if necessary
   ```

## Best Practices

### 1. SRGB planning

**Recommendations**:

- **Single SRGB**: Use the same SRGB on all nodes in the SR domain
- **Range reservation**: Do not use the entire SRGB; leave a reserve for expansion
- **Avoid conflicts**: The SRGB must not overlap with dynamic labels (LDP: 1024+)
- **Documentation**: Maintain a table of SID index allocations

**Recommended ranges**:

```
Small networks (<100 nodes):  SRGB 16000-17999 (2000 labels)
Medium networks (<1000 nodes): SRGB 16000-23999 (8000 labels)
Large networks (>1000 nodes):  SRGB 16000-80999 (65000 labels)
```

**Example SID documentation**:

```
Node-SID Allocation Table:
--------------------------
Index    Node         Loopback      Location
-----    ----         --------      --------
1        R1-MSK-PE1   10.0.0.1      Moscow DC1
2        R2-MSK-PE2   10.0.0.2      Moscow DC1
3        R3-SPB-PE1   10.0.0.3      SPb DC2
10-19    Core P       10.0.10.x     Core network
100-199  Anycast      10.100.x.x    Service groups
```

### 2. Naming and numbering

**Loopback addressing**:

```bash
# Use a consistent scheme for loopbacks
# Example: 10.0.<POD>.<NODE>

# Moscow Pod 0
set interfaces loopback lo address 10.0.0.1/32  # PE1
set interfaces loopback lo address 10.0.0.2/32  # PE2

# SPb Pod 1
set interfaces loopback lo address 10.0.1.1/32  # PE1
set interfaces loopback lo address 10.0.1.2/32  # PE2

# Core
set interfaces loopback lo address 10.0.10.1/32  # P1
set interfaces loopback lo address 10.0.10.2/32  # P2
```

**SID indexes**:

```bash
# Tie the index to the last octet of the loopback
# 10.0.0.1 -> Index 1
# 10.0.0.2 -> Index 2
# 10.0.10.1 -> Index 101
# 10.0.10.2 -> Index 102
```

### 3. Protecting the network with TI-LFA

**Enable TI-LFA everywhere**:

```bash
# On all nodes and interfaces
set protocols isis fast-reroute lfa local priority-limit critical
set protocols isis interface eth1 fast-reroute lfa local enable
set protocols isis interface eth2 fast-reroute lfa local enable
```

**Prioritizing critical prefixes**:

```bash
# Use priority-limit to protect only important routes
set protocols isis fast-reroute lfa local priority-limit high
```

**Failover testing**:

```bash
# Test resiliency regularly
# Bring an interface down and check the convergence time
set interfaces ethernet eth1 disable
commit
# Check the routing table and MPLS FIB
show ip route
show mpls table
# Bring it back up
delete interfaces ethernet eth1 disable
commit
```

### 4. Monitoring and logging

**Enable detailed SR logging**:

```bash
# IS-IS logging with SR
set system syslog global facility local7 level debug

# Logging to a separate file
set system syslog file isis facility local7 level info
```

**Monitoring key metrics**:

```bash
# Create a monitoring script
cat > /config/scripts/sr-monitor.sh << 'EOF'
#!/bin/bash
# SR Monitoring Script

echo "=== SR Status ==="
date

echo "=== SRGB ==="
cli-shell-api showConfig protocols segment-routing

echo "=== SR Prefix-SIDs ==="
vtysh -c "show isis segment-routing prefix-sids"

echo "=== MPLS Table ==="
vtysh -c "show mpls table"

echo "=== TI-LFA Summary ==="
vtysh -c "show isis fast-reroute summary"

echo "=== MPLS Statistics ==="
ip -s -s link show | grep -A 10 eth
EOF

chmod +x /config/scripts/sr-monitor.sh

# Run via cron every 5 minutes
set system task-scheduler task sr-monitor executable path /config/scripts/sr-monitor.sh
set system task-scheduler task sr-monitor interval 5m
```

**SNMP monitoring**:

```bash
# Enable SNMP for the MPLS MIB
set service snmp community public authorization ro
set service snmp community public network 192.168.100.0/24
```

### 5. SR security

**IGP authentication**:

```bash
# IS-IS authentication
set protocols isis area-password plaintext-password 'SECRET_AREA_PWD'
set protocols isis domain-password plaintext-password 'SECRET_DOMAIN_PWD'

# OSPF authentication
set protocols ospf area 0 authentication md5
set protocols ospf area 0 area-type stub
set protocols ospf interface eth1 authentication md5 key-id 1 md5-key 'SECRET_KEY'
```

**Filtering and Protection**:

```bash
# Limit the maximum label stack depth
set protocols segment-routing maximum-label-stack-depth 5

# Filter labeled BGP routes (for VPN)
set policy route-map VPN-IMPORT rule 10 match ip address prefix-list ALLOWED-PREFIXES
```

**Rate limiting for the Control Plane**:

```bash
# Protect the CPU from MPLS flooding
set firewall group network-group MPLS-SOURCES network 10.0.0.0/8
set firewall name PROTECT_CP rule 10 source group network-group MPLS-SOURCES
set firewall name PROTECT_CP rule 10 protocol 89
set firewall name PROTECT_CP rule 10 action accept
set firewall name PROTECT_CP rule 10 limit rate 100/second
```

### 6. Performance optimization

**Tuning IGP timers**:

```bash
# IS-IS fast convergence
set protocols isis fast-reroute lfa local load-sharing disable
set protocols isis spf-interval 1

# OSPF fast convergence
set protocols ospf parameters spf throttle delay 50
set protocols ospf parameters spf throttle initial-holdtime 50
set protocols ospf parameters spf throttle max-holdtime 5000
```

**MPLS optimization**:

```bash
# Disable TTL propagation for security
set protocols mpls no-propagate-ttl

# Use hardware offload (if supported)
# Depends on the platform
```

**Load balancing**:

```bash
# ECMP for SR paths
set protocols isis max-paths 4

# OSPF ECMP
set protocols ospf parameters ecmp 4
```

### 7. Scaling

**Hierarchical architecture**:

```bash
# Use different areas/levels for scaling
# IS-IS: Level-1 for access, Level-2 for core
set protocols isis level level-1-2
set protocols isis interface eth0 level level-1
set protocols isis interface eth1 level level-2

# OSPF: Multiple areas
set protocols ospf area 0 network 10.0.0.0/24
set protocols ospf area 1 network 10.1.0.0/16
```

**Route summarization**:

```bash
# OSPF summary on the ABR
set protocols ospf area 1 range 10.1.0.0/16

# IS-IS summary at the level boundary
set protocols isis redistribute ipv4 level-2 route-map SUMMARIZE
```

**Anycast SID for load balancing**:

```bash
# Assign the same Anycast SID on multiple PEs
# PE1:
set protocols ospf segment-routing prefix 10.100.1.0/32 index value 101

# PE2:
set protocols ospf segment-routing prefix 10.100.1.0/32 index value 101

# Traffic will be balanced between PE1 and PE2
```

### 8. Migrating to SR

**Gradual migration from LDP**:

```bash
# Stage 1: Enable SR in parallel with LDP
set protocols segment-routing ...
set protocols mpls ldp ...

# Stage 2: Verify that SR paths work
show mpls table

# Stage 3: Move critical services to SR
# (reconfigure BGP VPN, L2VPN, etc.)

# Stage 4: Disable LDP
delete protocols mpls ldp
commit
```

**Testing before production**:

```bash
# Build a test environment (GNS3, EVE-NG)
# Test all scenarios:
# - Normal operation
# - Link failure
# - Node failure
# - SR policy application
# - TI-LFA convergence
```

### 9. Backup and recovery

**Regular configuration backup**:

```bash
# Automatic backup
cat > /config/scripts/backup-config.sh << 'EOF'
#!/bin/bash
BACKUP_DIR="/config/backups"
DATE=$(date +%Y%m%d-%H%M%S)
FILENAME="config-${DATE}.boot"

mkdir -p $BACKUP_DIR
cli-shell-api showConfig > $BACKUP_DIR/$FILENAME

# Keep only the last 10 backups
ls -t $BACKUP_DIR/config-*.boot | tail -n +11 | xargs rm -f
EOF

chmod +x /config/scripts/backup-config.sh

# Daily backup at 02:00
set system task-scheduler task backup-config executable path /config/scripts/backup-config.sh
set system task-scheduler task backup-config interval 1d
```

**Documenting changes**:

```bash
# Commit with a comment
commit comment "Added SR for new DC2 PE nodes"

# View the history
show system commit

# Roll back to the previous configuration
rollback 1
commit
```

### 10. Integration with external systems

**Integration with NetBox/IPAM**:

```python
# Script to synchronize SIDs from NetBox
import pynetbox
import paramiko

nb = pynetbox.api('https://netbox.example.com', token='TOKEN')
devices = nb.dcim.devices.filter(role='router', site='moscow')

for device in devices:
    loopback_ip = device.primary_ip4.address.split('/')[0]
    sid_index = int(loopback_ip.split('.')[-1])

    # Generate the SR configuration
    config = f"""
    set protocols ospf segment-routing prefix {loopback_ip}/32 index value {sid_index}
    """

    # Apply over SSH (use with caution!)
    # ... SSH connection and configuration ...
```

**Prometheus monitoring**:

```bash
# Export MPLS metrics via the node_exporter textfile collector
cat > /config/scripts/mpls-metrics.sh << 'EOF'
#!/bin/bash
METRICS_FILE="/var/lib/node_exporter/textfile_collector/mpls.prom"

# MPLS table size
MPLS_TABLE_SIZE=$(vtysh -c "show mpls table" | grep -c "^[0-9]")
echo "vyos_mpls_table_size $MPLS_TABLE_SIZE" > $METRICS_FILE

# SR Prefix-SID count
SR_PREFIX_COUNT=$(vtysh -c "show isis segment-routing prefix-sids" | grep -c "^  ")
echo "vyos_sr_prefix_sid_count $SR_PREFIX_COUNT" >> $METRICS_FILE

# TI-LFA protected routes
TILFA_PROTECTED=$(vtysh -c "show isis fast-reroute summary" | grep -oP '\d+(?= prefixes protected)')
echo "vyos_tilfa_protected_prefixes ${TILFA_PROTECTED:-0}" >> $METRICS_FILE
EOF

chmod +x /config/scripts/mpls-metrics.sh

# Every minute
set system task-scheduler task mpls-metrics executable path /config/scripts/mpls-metrics.sh
set system task-scheduler task mpls-metrics interval 1m
```

## Limitations of the current implementation

VyOS implements Segment Routing on top of FRRouting, which has the following limitations:

### Current limitations (VyOS 1.4/1.5)

1. **Binding SID**: Not supported
2. **SRLB (Segment Routing Local Block)**: Configuration is not available
3. **Multiple SRGBs**: Only a single SRGB is supported
4. **Flex-Algo**: Not implemented (SPF algorithm only)
5. **Level Redistribution**: Not supported for SR in IS-IS
6. **SR-TE (Traffic Engineering)**: Limited support, no dynamic SR policies
7. **SRv6**: Not implemented in VyOS (available in upstream FRR)
8. **SR-MPLS with BGP-LS**: Not integrated
9. **PCE (Path Computation Element)**: No support for an external PCE

### Workarounds

**For SR-TE policies**:

```bash
# Use static routes with an explicit SID list
set protocols static route 10.5.0.0/16 segments 16002 16005 16008
```

**For Anycast (instead of Binding SID)**:

```bash
# Assign the same Prefix-SID on multiple nodes
# Node 1:
set protocols ospf segment-routing prefix 10.100.1.1/32 index value 100
# Node 2:
set protocols ospf segment-routing prefix 10.100.1.1/32 index value 100
```

## Additional resources

### Official documentation

- [VyOS Segment Routing Documentation](https://docs.vyos.io/en/latest/configuration/protocols/segment-routing.html)
- [FRRouting Segment Routing](http://docs.frrouting.org/en/latest/segment-routing.html)
- [IETF Segment Routing Architecture (RFC 8402)](https://datatracker.ietf.org/doc/html/rfc8402)
- [Segment Routing with IS-IS (RFC 8667)](https://datatracker.ietf.org/doc/html/rfc8667)
- [Segment Routing with OSPF (RFC 8665)](https://datatracker.ietf.org/doc/html/rfc8665)

### Useful RFCs

- **RFC 8402**: Segment Routing Architecture
- **RFC 8660**: Segment Routing with MPLS data plane
- **RFC 8665**: OSPF Extensions for Segment Routing
- **RFC 8666**: OSPFv3 Extensions for Segment Routing
- **RFC 8667**: IS-IS Extensions for Segment Routing
- **RFC 8919**: IS-IS Application-Specific Link Attributes
- **RFC 9256**: Segment Routing Policy Architecture

### Tools and utilities

```bash
# VyOS CLI
show segment-routing
show isis segment-routing
show ip ospf segment-routing
show mpls table
show mpls fib

# FRRouting vtysh
vtysh -c "show segment-routing srv6 locator"
vtysh -c "show isis segment-routing node"
vtysh -c "show mpls table"

# Linux kernel MPLS
ip -M route show
ip -M route show table all

# MPLS traceroute
traceroute -m 30 -M mpls <destination>
```

### Example configurations

Complete working examples are available in:

- [VyOS Examples Repository](https://github.com/vyos/vyos-documentation/tree/current/docs/configuration/protocols/examples)
- [FRR SR Examples](https://github.com/FRRouting/frr/tree/master/tests/topotests/isis_sr_topo1)

## Conclusion

Segment Routing is a modern approach to traffic management in MPLS networks, providing:

- Simplified operational management (without LDP, RSVP-TE)
- Flexible traffic engineering via SR policies
- Fast convergence with TI-LFA (<50ms failover)
- Scalability for large networks
- Compatibility with existing MPLS infrastructure

VyOS with SR-MPLS support through FRRouting enables you to build modern network solutions for cloud providers, ISPs, and enterprise networks. With proper SRGB planning, a sound architecture, and adherence to best practices, Segment Routing provides a reliable and flexible platform for critical network services.

Regular monitoring, failover-scenario testing, and configuration documentation are key factors in the successful operation of SR-based networks.

---

**Last updated**: 2025-10-15
**Document version**: 1.0
**Applies to**: VyOS 1.4 (Sagitta), VyOS 1.5 (Circinus)