# Updates - Upgrading the VyOS System

> Upgrading VyOS - checking versions, managing images, and performing update and rollback procedures with examples for Yandex Cloud and VK Cloud

Source: https://opennix.org/en/docs/vyos/system/vyos-updates/


## Updates - Upgrading the VyOS System

This page describes the VyOS update mechanism, including automatic update checks, system image management, and the procedures for upgrading and rolling back to previous versions.

## Overview

VyOS uses an image-based update mechanism, which provides:

- **Safety**: The ability to roll back to the previous version if problems occur
- **Predictability**: A complete replacement of system files, with no partial updates
- **Isolation**: Each version is stored as a separate image
- **Fast rollback**: Switching between versions via a reboot
- **Testing**: The ability to validate a new version before making the final switch

### System image architecture

VyOS stores several system images on a single device:

```
/boot/
├── 1.5-rolling-202401150023/    # Current version (default)
├── 1.5-rolling-202312250024/    # Previous version
└── 1.4.0-sagitta/               # Older LTS version
```

**Key features:**

- All images share a common `/config/` partition
- The configuration is preserved when switching between images
- At boot time you can select any installed image
- Deleting an image does not affect the configuration

### Types of VyOS updates

**Rolling Release (1.5+):**
- Continuous updates with new features
- Daily nightly builds
- Recommended for testing and lab environments
- Requires regular monitoring for updates

**LTS (Long Term Support):**
- Stable releases (for example, 1.4 Sagitta)
- Security fixes and critical bug fixes only
- Recommended for production
- A predictable update cycle

## Checking the current version

### Version-check commands

```bash
# Show the current system version
show version

# Show all installed images
show system image

# Show detailed system information
show version all
```

### Example output of show version

```bash
vyos@router:~$ show version
Version:          VyOS 1.5-rolling-202401150023
Release train:    current
Release flavor:   generic

Built by:         autobuild@vyos.net
Built on:         Mon 15 Jan 2024 00:23 UTC
Build UUID:       a7b3c9d2-e4f5-6789-0abc-def123456789
Build commit ID:  abcdef123456

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (Q35 + ICH9, 2009)
Hardware S/N:     unknown
Hardware UUID:    12345678-1234-1234-1234-123456789abc

Copyright:        VyOS maintainers and contributors
```

### Example output of show system image

```bash
vyos@router:~$ show system image
The system currently has the following image(s) installed:

   1: 1.5-rolling-202401150023 (default boot) (running image)
   2: 1.5-rolling-202312250024
   3: 1.4.0-sagitta

Total images: 3
```

## Configuring automatic update checks

### Enabling automatic checks

VyOS can automatically check for new versions and notify the administrator.

```bash
# Enable automatic update checks
configure
set system update-check auto-check
commit
save
exit
```

### Configuring the update-check URL

```bash
configure

# URL for the rolling release (1.5+)
set system update-check url 'https://raw.githubusercontent.com/vyos/vyos-rolling-nightly-builds/main/version.json'

# Enable automatic checks
set system update-check auto-check

commit
save
exit
```

### Checking for available updates

```bash
# Manually check for available updates
show system updates
```

### Example output of show system updates

```bash
vyos@router:~$ show system updates
Current version: 1.5-rolling-202312220023

Update available: 1.5-rolling-202401150023
Update URL: https://github.com/vyos/vyos-rolling-nightly-builds/releases/download/1.5-rolling-202401150023/1.5-rolling-202401150023-amd64.iso

To update, use: add system image <URL>
```

### Disabling automatic checks

```bash
configure
delete system update-check auto-check
commit
save
exit
```

## Adding a new system image

### Installing an image from a URL

```bash
# Add the latest available version
add system image latest

# Add an image from a specific URL
add system image https://github.com/vyos/vyos-rolling-nightly-builds/releases/download/1.5-rolling-202401150023/1.5-rolling-202401150023-amd64.iso

# Add an image from a local file
add system image /tmp/vyos-1.5-rolling-202401150023-amd64.iso
```

### The image installation process

```bash
vyos@router:~$ add system image latest
Trying to fetch latest version from https://github.com/vyos/vyos-rolling-nightly-builds/releases/download/1.5-rolling-202401150023/1.5-rolling-202401150023-amd64.iso

Checking for available updates...
Found new version: 1.5-rolling-202401150023

Do you want to continue? [y/N] y

Downloading image...
######################################################################## 100.0%

Checking image... OK

What would you like to name this image? [1.5-rolling-202401150023]:
# Press Enter for the default name or enter your own

Installing new image...
Copying files... Done

Do you want to set the new image as the default boot image? [y/N] y
# Choose 'y' to have the new image boot by default

Image installation complete.

Do you want to reboot now? [y/N] n
# You can postpone the reboot to finish ongoing tasks
```

### Installing an image with a custom name

```bash
vyos@router:~$ add system image https://example.com/vyos-custom.iso

What would you like to name this image? [vyos-custom]: production-20240115
# Convenient for identifying images

Installing new image...
Copying files... Done

Image 'production-20240115' installed successfully.
```

## Managing system images

### Setting the default image

```bash
# Set the image to boot by default
set system image default-boot 1.5-rolling-202401150023

# Or interactively
vyos@router:~$ set system image default-boot
Please specify image name:
   1: 1.5-rolling-202401150023
   2: 1.5-rolling-202312250024
   3: 1.4.0-sagitta
Select image [1-3]: 1

Default boot image set to: 1.5-rolling-202401150023
```

### Renaming an image

```bash
# Rename an image for easier identification
rename system image 1.5-rolling-202401150023 to production-current
rename system image 1.5-rolling-202312250024 to production-previous
```

### Deleting an image

```bash
# Delete an unused image to free up space
delete system image 1.4.0-sagitta

# Or interactively
vyos@router:~$ delete system image
The following images are available for deletion:
   1: 1.5-rolling-202312250024
   2: 1.4.0-sagitta

Select image to delete [1-2]: 2

Warning: This will permanently delete image '1.4.0-sagitta'
Continue? [y/N] y

Deleting image... Done
```

**Important:**
- You cannot delete the currently booted image (running image)
- You cannot delete the default image (default boot) without setting a new one first
- Deleting an image does not affect the configuration in `/config/`

### Checking disk space usage

```bash
# Check the space used by images
show system storage

# Detailed information
df -h
```

```bash
vyos@router:~$ show system storage
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2       9.5G  4.2G  4.8G  47% /
tmpfs           489M     0  489M   0% /dev/shm
/dev/sda1       497M  125M  372M  26% /boot

Images storage usage:
1.5-rolling-202401150023: 1.2G
1.5-rolling-202312250024: 1.2G
1.4.0-sagitta: 950M
Total: 3.35G
```

## The VyOS update procedure

### Preparing for the update

**Step 1: Create a configuration backup**

```bash
# Save the current configuration
save /config/backup-before-update-$(date +%Y%m%d).config

# Copy to an external server (optional)
scp /config/backup-before-update-$(date +%Y%m%d).config user@backup-server:/backups/
```

**Step 2: Check the current state**

```bash
# Check the current version
show version

# Check the installed images
show system image

# Check available space
show system storage

# Check active sessions
show system connections

# Check running processes
show system processes
```

**Step 3: Notify users**

```bash
# Send a warning message to all connected users
wall "WARNING: A system update is scheduled in 10 minutes. Please save your work."

# Or via the system login banner
configure
set system login banner pre-login "The system will be rebooted for an update at 18:00 MSK"
commit
save
exit
```

### Performing the update

**Step 1: Download and install the new image**

```bash
# Check for available updates
show system updates

# Install the new image
add system image latest

# When prompted:
# 1. Confirm the download: y
# 2. Specify the image name (or press Enter for the default)
# 3. Set as default boot: y
# 4. Reboot now: n (postpone until the maintenance window)
```

**Step 2: Verify the installation**

```bash
# Make sure the image is installed
show system image

# Verify that the new image is set as default
# The output should show (default boot) next to the new image
```

**Step 3: Scheduled reboot**

```bash
# Schedule a reboot (in 5 minutes)
shutdown reboot +5 "Rebooting to apply the VyOS update"

# Or an immediate reboot
reboot now

# Or at a specific time
shutdown reboot 18:00 "Scheduled VyOS update"
```

**Step 4: Verify after the reboot**

```bash
# Verify that the new version has booted
show version

# Verify that the configuration has been applied
show configuration

# Check network reachability
ping 8.8.8.8

# Check that services are working
show interfaces
show protocols
show firewall
```

### Quick update procedure (single command)

```bash
# For experienced administrators - an automated update
add system image latest && reboot now
```

**Caution**: Use only in test environments or when you fully understand the risks.

## Rolling back to a previous version (Rollback)

### Reasons for a rollback

- Hardware compatibility problems were discovered
- Critical bugs in the new version
- Performance problems
- Incompatibility with the existing configuration
- The need for additional testing

### Rollback methods

#### Method 1: Selecting an image via GRUB (at boot)

**When to use**: When the system cannot boot or there are critical problems

**Procedure**:

1. Reboot the system: `reboot`
2. When the GRUB menu appears (press ESC if it boots automatically)
3. Select the entry with the previous VyOS version
4. Press Enter

**Note**: If the GRUB menu does not appear, press ESC during boot.

#### Method 2: Changing the default boot image

**When to use**: When the system is running stably but you need to return to the previous version

**Procedure**:

```bash
# View the available images
show system image

# Output:
# 1: 1.5-rolling-202401150023 (default boot) (running image)
# 2: 1.5-rolling-202312250024
# 3: 1.4.0-sagitta

# Set the previous image as the default
set system image default-boot 1.5-rolling-202312250024

# Reboot the system
reboot now
```

#### Method 3: Interactive rollback

```bash
# Launch the interactive rollback wizard
vyos@router:~$ set system image default-boot

Please specify image name:
   1: 1.5-rolling-202401150023 (current default)
   2: 1.5-rolling-202312250024
   3: 1.4.0-sagitta

Select image [1-3]: 2

Default boot image set to: 1.5-rolling-202312250024

Reboot now to apply changes? [y/N] y
```

### Verification after a rollback

```bash
# Check the version after the reboot
show version

# Make sure the configuration was applied correctly
show configuration

# Check that critical services are working
show interfaces
show protocols
show firewall
show nat

# Check the logs for errors
show log tail 100
```

### Automatic rollback via a watchdog

For critical systems, you can configure an automatic rollback on loss of connectivity:

```bash
#!/bin/bash
# Automatic rollback script for when problems occur
# Placed on an external monitoring server

VYOS_HOST="192.168.1.1"
PING_TIMEOUT=30
MAX_FAILURES=3

failures=0

while true; do
    if ! ping -c 1 -W $PING_TIMEOUT $VYOS_HOST > /dev/null 2>&1; then
        ((failures++))
        echo "Ping failed ($failures/$MAX_FAILURES)"

        if [ $failures -ge $MAX_FAILURES ]; then
            echo "Critical: VyOS not responding. Manual intervention required!"
            # Send an alert to the administrator
            # mail -s "VyOS DOWN" admin@example.com < /dev/null
        fi
    else
        failures=0
    fi

    sleep 60
done
```

## Examples for cloud platforms

### Example 1: Yandex Cloud - Automated update checks

In Yandex Cloud, VyOS can automatically check for updates and notify via cloud monitoring.

#### Configuration for Yandex Cloud

```bash
configure

# Enable automatic update checks
set system update-check auto-check
set system update-check url 'https://raw.githubusercontent.com/vyos/vyos-rolling-nightly-builds/main/version.json'

# Configure syslog to send to Yandex Cloud Logging
set system syslog host 10.128.0.100 facility all level info
set system syslog host 10.128.0.100 facility all protocol tcp

# Configure SNMP for monitoring (if used)
set service snmp community yc-monitoring authorization ro
set service snmp listen-address 10.128.0.10

commit
save
exit
```

#### Update-check script for Yandex Cloud

```bash
#!/bin/vbash
# /config/scripts/yc-check-updates.sh
# Automatic update check with logging to Yandex Cloud

source /opt/vyatta/etc/functions/script-template

LOG_FILE="/var/log/vyos-update-check.log"
METADATA_URL="http://169.254.169.254/latest/meta-data"

# Get instance information
INSTANCE_ID=$(curl -s $METADATA_URL/instance-id)
ZONE=$(curl -s $METADATA_URL/placement/availability-zone)

echo "[$(date)] Checking updates for instance $INSTANCE_ID in zone $ZONE" >> $LOG_FILE

# Check for updates
UPDATE_INFO=$(cli-shell-api showCfg system update-check)

if [ -n "$UPDATE_INFO" ]; then
    echo "[$(date)] Update check enabled" >> $LOG_FILE

    # Run the check (the show command is unavailable in scripts, so use the API)
    # In production, use the VyOS API or a wrapper

    logger -t vyos-updates "Update check completed for $INSTANCE_ID"
else
    echo "[$(date)] Update check not configured" >> $LOG_FILE
fi

# Check disk space usage
DISK_USAGE=$(df -h / | tail -1 | awk '{print $5}' | sed 's/%//')

if [ $DISK_USAGE -gt 80 ]; then
    logger -t vyos-updates "WARNING: Disk usage is ${DISK_USAGE}% - consider removing old images"
    echo "[$(date)] WARNING: Disk usage ${DISK_USAGE}%" >> $LOG_FILE
fi

echo "[$(date)] Update check completed" >> $LOG_FILE
```

#### Configuring the task scheduler

```bash
configure

# Run the update check every day at 2:00 UTC
set system task-scheduler task check-updates executable path '/config/scripts/yc-check-updates.sh'
set system task-scheduler task check-updates interval 1d
set system task-scheduler task check-updates start-time '02:00'

commit
save
exit
```

#### Creating a snapshot before an update in Yandex Cloud

```bash
#!/bin/bash
# yc-create-snapshot-before-update.sh
# Script to create a disk snapshot before a VyOS update

INSTANCE_NAME="vyos-router-01"
DISK_NAME="vyos-boot-disk"
SNAPSHOT_NAME="vyos-backup-$(date +%Y%m%d-%H%M%S)"

# Requires the yc CLI to be installed and configured
yc compute snapshot create \
  --name "$SNAPSHOT_NAME" \
  --disk-name "$DISK_NAME" \
  --description "Automatic snapshot before VyOS update" \
  --labels "instance=$INSTANCE_NAME,type=pre-update"

echo "Snapshot $SNAPSHOT_NAME created successfully"

# Once the snapshot is created successfully, you can update VyOS
ssh vyos@$INSTANCE_NAME "add system image latest"
```

### Example 2: VK Cloud - Staged deployment with rollback

VK Cloud lets you create multiple instances for a blue-green deployment.

#### Scenario: Blue-Green update

**Architecture**:
- Blue environment: The current production version (VyOS 1.4)
- Green environment: The new version being tested (VyOS 1.5)
- A load balancer switches traffic between the environments

**Preparing the Green environment**:

```bash
# On a new VK Cloud instance
configure

# Basic configuration
set system host-name vyos-router-green
set system domain-name vkcloud.internal

# Check the current version
show version

# If an update is required - install the latest
exit

# Install the image
add system image latest

# Copy the configuration from Blue
scp vyos@blue-instance:/config/config.boot /tmp/blue-config.boot

# Load the configuration
configure
load /tmp/blue-config.boot

# Change the hostname for Green
set system host-name vyos-router-green

commit
save
exit
```

**Testing the Green environment**:

```bash
#!/bin/bash
# vkcloud-test-green-env.sh
# Testing the new environment before switching traffic

GREEN_IP="10.0.1.101"
TESTS_PASSED=0
TESTS_FAILED=0

echo "Testing Green environment at $GREEN_IP"

# Test 1: Ping connectivity
if ping -c 5 $GREEN_IP > /dev/null 2>&1; then
    echo "PASS: Ping connectivity"
    ((TESTS_PASSED++))
else
    echo "FAIL: Ping connectivity"
    ((TESTS_FAILED++))
fi

# Test 2: SSH availability
if ssh -o ConnectTimeout=5 vyos@$GREEN_IP "show version" > /dev/null 2>&1; then
    echo "PASS: SSH access"
    ((TESTS_PASSED++))
else
    echo "FAIL: SSH access"
    ((TESTS_FAILED++))
fi

# Test 3: Routing table
if ssh vyos@$GREEN_IP "show ip route" | grep -q "default"; then
    echo "PASS: Default route present"
    ((TESTS_PASSED++))
else
    echo "FAIL: Default route missing"
    ((TESTS_FAILED++))
fi

# Test 4: NAT rules
if ssh vyos@$GREEN_IP "show nat source rules" | grep -q "rule"; then
    echo "PASS: NAT rules configured"
    ((TESTS_PASSED++))
else
    echo "FAIL: NAT rules missing"
    ((TESTS_FAILED++))
fi

# Test 5: Firewall rules
if ssh vyos@$GREEN_IP "show firewall" | grep -q "rule"; then
    echo "PASS: Firewall rules configured"
    ((TESTS_PASSED++))
else
    echo "FAIL: Firewall rules missing"
    ((TESTS_FAILED++))
fi

echo ""
echo "Results: $TESTS_PASSED passed, $TESTS_FAILED failed"

if [ $TESTS_FAILED -eq 0 ]; then
    echo "All tests passed - safe to proceed with traffic switchover"
    exit 0
else
    echo "Tests failed - do not switch traffic, investigate issues"
    exit 1
fi
```

**Switching traffic (via the VK Cloud Load Balancer)**:

```bash
#!/bin/bash
# vkcloud-switch-to-green.sh
# Switching traffic to the Green environment

# Check the tests
if ! ./vkcloud-test-green-env.sh; then
    echo "Pre-flight tests failed. Aborting switchover."
    exit 1
fi

# Gradual switchover (canary deployment)
echo "Phase 1: Switching 10% traffic to Green..."
# Configure the Load Balancer for 10% of traffic to Green
# (done via the VK Cloud API or the management console)

sleep 300  # 5 minutes of monitoring

echo "Phase 2: Switching 50% traffic to Green..."
# 50% of traffic to Green

sleep 300

echo "Phase 3: Switching 100% traffic to Green..."
# 100% of traffic to Green

echo "Switchover complete. Blue environment remains as rollback option."
```

**Rollback procedure**:

```bash
#!/bin/bash
# vkcloud-rollback-to-blue.sh
# Rollback to the Blue environment when problems occur

echo "EMERGENCY ROLLBACK: Switching all traffic to Blue environment"

# Immediately switch 100% of traffic to Blue
# (via the VK Cloud Load Balancer API)

echo "All traffic redirected to Blue. Green environment available for investigation."

# Send a notification to the team
wall "ROLLBACK EXECUTED: All traffic returned to Blue environment"
```

#### Automatic monitoring and rollback

```bash
#!/bin/bash
# vkcloud-auto-rollback-monitor.sh
# Automatic rollback when problems are detected

GREEN_IP="10.0.1.101"
BLUE_IP="10.0.1.100"
FAILURE_THRESHOLD=3
CHECK_INTERVAL=30

failures=0

while true; do
    # Check the availability of Green
    if ! ping -c 3 -W 5 $GREEN_IP > /dev/null 2>&1; then
        ((failures++))
        echo "$(date): Green environment check failed ($failures/$FAILURE_THRESHOLD)"

        if [ $failures -ge $FAILURE_THRESHOLD ]; then
            echo "$(date): CRITICAL - Initiating automatic rollback to Blue"
            ./vkcloud-rollback-to-blue.sh

            # Send a critical notification
            curl -X POST https://monitoring.vkcloud.example/alert \
                -d "severity=critical&message=Auto-rollback to Blue executed"

            # Stop monitoring after the rollback
            exit 1
        fi
    else
        # Reset the counter on a successful check
        if [ $failures -gt 0 ]; then
            echo "$(date): Green environment recovered"
        fi
        failures=0
    fi

    sleep $CHECK_INTERVAL
done
```

### Example 3: Hybrid cloud environment - Multi-cloud update strategy

```bash
#!/bin/bash
# multi-cloud-update-strategy.sh
# Updating VyOS routers in a multi-cloud environment

# Routers in different clouds
YANDEX_ROUTERS="10.128.0.10 10.128.0.11"
VK_ROUTERS="10.0.1.10 10.0.1.11"
ONPREM_ROUTERS="192.168.1.10 192.168.1.11"

# Function to update a single router
update_router() {
    local ROUTER_IP=$1
    local ENV_NAME=$2

    echo "[$ENV_NAME] Updating router $ROUTER_IP"

    # Create a configuration backup
    ssh vyos@$ROUTER_IP "save /config/backup-$(date +%Y%m%d).config"

    # Download the backup to the local server
    scp vyos@$ROUTER_IP:/config/backup-$(date +%Y%m%d).config ./backups/$ENV_NAME-$ROUTER_IP-$(date +%Y%m%d).config

    # Update the image
    ssh vyos@$ROUTER_IP "add system image latest" < /dev/null

    # Schedule a reboot in 5 minutes (to allow cancellation)
    ssh vyos@$ROUTER_IP "shutdown reboot +5 'Automatic update'"

    echo "[$ENV_NAME] Router $ROUTER_IP scheduled for update"
}

# Update in stages: Yandex Cloud first
echo "Stage 1: Updating Yandex Cloud routers"
for ROUTER in $YANDEX_ROUTERS; do
    update_router $ROUTER "Yandex Cloud"
    sleep 600  # 10 minutes between routers
done

sleep 1800  # 30 minutes to verify

# Then VK Cloud
echo "Stage 2: Updating VK Cloud routers"
for ROUTER in $VK_ROUTERS; do
    update_router $ROUTER "VK Cloud"
    sleep 600
done

sleep 1800

# Finally, on-premise
echo "Stage 3: Updating on-premise routers"
for ROUTER in $ONPREM_ROUTERS; do
    update_router $ROUTER "On-Premise"
    sleep 600
done

echo "All routers updated. Monitor for next 24 hours."
```

## Verification and monitoring commands

### Checking the version and images

```bash
# Current version
show version

# Short version (number only)
show version brief

# All installed images
show system image

# Detailed information
show version all
```

### Checking for updates

```bash
# Check for available updates
show system updates

# Show the update-check configuration
show configuration system update-check

# Show the update-check URL
show configuration system update-check url
```

### Monitoring disk space

```bash
# Overall usage
show system storage

# Detailed by filesystem
df -h

# Size of each image
du -sh /boot/*/

# Top 10 largest files in /config
du -ah /config | sort -rh | head -10
```

### Checking the boot history

```bash
# Recent reboots
last reboot

# Uptime of the current session
show system uptime

# Date of the last boot
who -b
```

### Update logs

```bash
# Image installation logs
show log | match image

# System logs
show log tail 100

# Boot logs
show log boot

# Search for errors during an update
show log | match -i "error\|fail" | match image
```

## Troubleshooting

### Problem 1: Not enough space to install a new image

**Symptoms**:
```
Error: Not enough disk space to install new image
Required: 1.5G, Available: 800M
```

**Diagnostics**:

```bash
# Check available space
show system storage

# Show all images
show system image

# Size of each image
du -sh /boot/*/
```

**Solution**:

```bash
# Delete old, unused images
delete system image 1.4.0-sagitta

# Clear old logs
sudo find /var/log -type f -name "*.gz" -delete
sudo find /var/log -type f -name "*.old" -delete

# Clear old configuration backups
sudo rm /config/backup-*.config

# Check the freed-up space
show system storage

# Retry the installation
add system image latest
```

### Problem 2: The new image hangs during boot

**Symptoms**:
- The system hangs on the boot screen
- GRUB shows the new image, but the boot does not complete
- Kernel panic messages appear

**Solution via GRUB**:

1. Reboot the system (physically or via IPMI/KVM)
2. Press ESC when GRUB appears
3. Select the previously working image
4. After a successful boot:

```bash
# Set the old image as the default
set system image default-boot 1.5-rolling-202312250024

# Delete the problematic image
delete system image 1.5-rolling-202401150023

# Check the logs for diagnostics
show log boot
show log kernel
```

### Problem 3: The configuration is not applied after an update

**Symptoms**:
- After the update and reboot, the configuration is not applied
- Interfaces are not configured
- Services are not started

**Diagnostics**:

```bash
# Check the current version
show version

# Check the configuration contents
show configuration

# Check the configuration file
cat /config/config.boot

# Check the boot logs
show log boot | match -i "error\|fail\|warn"
```

**Solution**:

```bash
# Load the configuration from a backup
configure
load /config/backup-before-update-20240115.config
commit
save
exit

# If no backup exists - restore a minimal configuration
configure

# Basic interface configuration
set interfaces ethernet eth0 address dhcp
set interfaces ethernet eth0 description 'WAN'

set interfaces ethernet eth1 address 192.168.1.1/24
set interfaces ethernet eth1 description 'LAN'

# NAT
set nat source rule 100 outbound-interface name 'eth0'
set nat source rule 100 source address 192.168.1.0/24
set nat source rule 100 translation address masquerade

commit
save
exit
```

### Problem 4: Error downloading the image from a URL

**Symptoms**:
```
Error: Failed to download image from URL
Connection timeout or HTTP error
```

**Diagnostics**:

```bash
# Check network connectivity
ping -c 5 github.com

# Check DNS
nslookup github.com

# Check the route to the internet
show ip route

# Check NAT (if used)
show nat source statistics

# Try to fetch the URL manually
curl -I https://github.com/vyos/vyos-rolling-nightly-builds/releases/download/1.5-rolling-202401150023/1.5-rolling-202401150023-amd64.iso
```

**Solution**:

```bash
# Option 1: Use plain HTTP instead of HTTPS (if possible)
add system image http://mirror.example.com/vyos-image.iso

# Option 2: Download the image to a local server
# On an external server with internet access:
wget https://github.com/vyos/vyos-rolling-nightly-builds/releases/download/1.5-rolling-202401150023/1.5-rolling-202401150023-amd64.iso

# Start an HTTP server
python3 -m http.server 8080

# On VyOS:
add system image http://192.168.1.100:8080/1.5-rolling-202401150023-amd64.iso

# Option 3: Transfer via SCP
# Copy the file to VyOS
scp vyos-image.iso vyos@router:/tmp/

# Install from the local file
add system image /tmp/vyos-image.iso
```

### Problem 5: Access to the system is lost after an update

**Symptoms**:
- The SSH connection drops after the reboot
- You cannot connect to the router
- Ping does not go through

**Diagnostics via the console** (KVM/IPMI/Serial):

```bash
# Log in via the console
# Check the interfaces
show interfaces

# Check the IP addresses
ip addr show

# Check routing
show ip route

# Check the firewall
show firewall
```

**Solution**:

```bash
# Via the console - restore network connectivity
configure

# Verify that the interfaces are configured
show interfaces

# If not - configure them again
set interfaces ethernet eth0 address 192.168.1.1/24

# Temporarily disable the firewall for diagnostics
set firewall all-ping enable

commit
exit

# Try the SSH connection again
```

**If the console is unavailable** - roll back via GRUB:
1. Reboot (hard reset)
2. Select the previous image in GRUB
3. After booting, set it as the default

### Problem 6: The GRUB menu does not appear

**Symptoms**:
- The system automatically boots the default image
- You cannot select a different image

**Solution**:

**Method 1: Configure the GRUB timeout**

Via SSH (before the problem occurs):
```bash
# Increase the GRUB timeout to allow selecting an image
sudo vi /boot/grub/grub.cfg

# Find the line:
# set timeout=0
# Change it to:
# set timeout=10

# Save and reboot
```

**Method 2: Press ESC during boot**

- When the VyOS logo appears, press ESC repeatedly
- The GRUB menu should appear even when timeout=0

**Method 3: Via the serial console**

```bash
# Configure the serial console for GRUB access
configure
set system console device ttyS0 speed 9600
commit
save
exit
```

## Best Practices

### 1. Back up before an update

**Always create a configuration backup:**

```bash
# Automatic backup with a timestamp
save /config/backup-$(date +%Y%m%d-%H%M%S).config

# Copy to an external server
scp /config/backup-$(date +%Y%m%d-%H%M%S).config user@backup-server:/vyos-backups/

# For critical systems - a disk snapshot (in the cloud)
# Yandex Cloud: create a snapshot via the web UI or API
# VK Cloud: create a snapshot via the management console
```

### 2. Test updates

**Never update production without testing:**

```bash
# Testing pipeline:
# 1. Lab environment (isolated network)
# 2. Dev environment (development)
# 3. Staging environment (pre-production)
# 4. Production (on the maintenance-window schedule)

# For each stage:
# - Install the image
# - Load the production configuration
# - Test all critical functions
# - Monitor performance
# - Only after success - move to the next stage
```

### 3. Document updates

**Keep an update log:**

```bash
# Example log entry
# /config/update-log.txt

2024-01-15 18:00 MSK - Update started
Router: vyos-router-01
Previous version: 1.5-rolling-202312220023
New version: 1.5-rolling-202401150023
Reason: Security patches
Executed by: admin@example.com
Backup: /backups/vyos-20240115-180000.config
Rollback plan: GRUB select previous image

2024-01-15 18:15 MSK - Update completed successfully
Post-update checks: PASSED
All services operational
```

### 4. Maintenance Windows

**Schedule updates during off-peak hours:**

```bash
# Define a maintenance window
# Examples:
# - Corporate network: 22:00-02:00 on weekdays
# - ISP: 03:00-05:00 on weekdays
# - Critical systems: weekends only

# Notify users in advance
configure
set system login banner pre-login "
MAINTENANCE WINDOW: 15.01.2024 22:00-02:00 MSK
VyOS update scheduled. Brief service interruption expected.
"
commit
save
exit
```

### 5. Staged Rollout

**For multiple routers - update gradually:**

```bash
# Week 1: Lab and Dev routers (10% fleet)
# Week 2: Staging and non-critical (30% fleet)
# Week 3: Half of production (50% fleet)
# Week 4: Remaining production (100% fleet)

# Between stages:
# - Monitor for 7 days
# - Collect feedback from users
# - Analyze logs and metrics
# - Halt the rollout if problems occur
```

### 6. Monitoring after an update

**Enhanced monitoring for the first 24-48 hours:**

```bash
#!/bin/bash
# post-update-monitoring.sh
# Extended monitoring after an update

ROUTER_IP="192.168.1.1"
CHECK_INTERVAL=60  # seconds
DURATION=86400     # 24 hours

end_time=$(($(date +%s) + DURATION))

while [ $(date +%s) -lt $end_time ]; do
    echo "=== $(date) ==="

    # CPU usage
    ssh vyos@$ROUTER_IP "show system cpu" | grep "CPU"

    # Memory usage
    ssh vyos@$ROUTER_IP "show system memory" | grep "Mem:"

    # Interface status
    ssh vyos@$ROUTER_IP "show interfaces" | grep -E "eth[0-9]|state"

    # Connection count
    ssh vyos@$ROUTER_IP "show system connections" | wc -l

    # Errors in logs
    ERRORS=$(ssh vyos@$ROUTER_IP "show log tail 100 | grep -i error" | wc -l)
    if [ $ERRORS -gt 0 ]; then
        echo "WARNING: $ERRORS errors detected in logs"
    fi

    sleep $CHECK_INTERVAL
done
```

### 7. Rollback Plan

**Always have a rollback plan ready:**

```bash
# Document rollback-plan.md

# ROLLBACK PLAN for the 15.01.2024 update

## Conditions for a rollback:
1. Critical services unavailable > 5 minutes
2. Loss of connectivity > 3 minutes
3. CPU/Memory usage > 90% for 10 minutes
4. Critical errors detected in the logs

## Rollback procedure:
1. SSH access: ssh vyos@192.168.1.1
2. Command: set system image default-boot 1.5-rolling-202312220023
3. Command: reboot now
4. Wait: 5 minutes
5. Verify: ping, SSH, services
6. Notify: the team about the rollback

## Escalation contacts:
- L1 Support: +7-xxx-xxx-xxxx
- L2 Engineering: engineering@example.com
- L3 Vendor Support: VyOS commercial support

## Backup location:
- Config: /backups/vyos-20240115.config
- Disk snapshot (Yandex Cloud): snapshot-20240115-180000
```

### 8. Versioning and labels

**Use clear names for images:**

```bash
# Instead of:
# 1.5-rolling-202401150023

# Use rename:
rename system image 1.5-rolling-202401150023 to production-current
rename system image 1.5-rolling-202312220023 to production-previous
rename system image 1.4.0-sagitta to lts-stable

# Result:
show system image
# 1: production-current (default boot) (running image)
# 2: production-previous
# 3: lts-stable
```

### 9. Automation via CI/CD

**For large infrastructures - automation:**

```yaml
# .gitlab-ci.yml - an example CI/CD pipeline for VyOS updates

stages:
  - backup
  - update
  - verify
  - rollback

variables:
  VYOS_ROUTERS: "10.0.1.10 10.0.1.11 10.0.1.12"
  IMAGE_URL: "https://downloads.example.com/vyos-latest.iso"

backup_configs:
  stage: backup
  script:
    - for router in $VYOS_ROUTERS; do
        ssh vyos@$router "save /config/backup-$(date +%Y%m%d).config";
        scp vyos@$router:/config/backup-$(date +%Y%m%d).config ./backups/;
      done
  artifacts:
    paths:
      - backups/

update_routers:
  stage: update
  script:
    - for router in $VYOS_ROUTERS; do
        ssh vyos@$router "add system image $IMAGE_URL";
        ssh vyos@$router "shutdown reboot +5";
        sleep 600;
      done
  dependencies:
    - backup_configs

verify_update:
  stage: verify
  script:
    - ./scripts/verify-routers.sh
  allow_failure: true

rollback_on_failure:
  stage: rollback
  when: on_failure
  script:
    - ./scripts/rollback-all-routers.sh
```

### 10. Security during updates

**Verify the authenticity of images:**

```bash
# Verify the image checksum
curl -O https://downloads.vyos.io/rolling/current/vyos-1.5-rolling-202401150023-amd64.iso
curl -O https://downloads.vyos.io/rolling/current/vyos-1.5-rolling-202401150023-amd64.iso.sha256

# Verify the checksum
sha256sum -c vyos-1.5-rolling-202401150023-amd64.iso.sha256

# Only install after a successful verification
add system image /tmp/vyos-1.5-rolling-202401150023-amd64.iso
```

## Update automation

### Automated update script with checks

```bash
#!/bin/bash
# auto-update-vyos.sh
# Fully automated update with checks

set -e

ROUTER_IP="192.168.1.1"
BACKUP_DIR="/backups/vyos"
LOG_FILE="/var/log/vyos-auto-update.log"
ADMIN_EMAIL="admin@example.com"

log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a $LOG_FILE
}

send_notification() {
    local subject="$1"
    local body="$2"
    echo "$body" | mail -s "$subject" $ADMIN_EMAIL
}

# Pre-flight checks
log "Starting pre-flight checks"

# Check SSH connectivity
if ! ssh -o ConnectTimeout=5 vyos@$ROUTER_IP "show version" > /dev/null 2>&1; then
    log "ERROR: Cannot connect to router"
    send_notification "VyOS Update FAILED" "Cannot connect to router $ROUTER_IP"
    exit 1
fi

# Check disk space
DISK_USAGE=$(ssh vyos@$ROUTER_IP "df -h / | tail -1 | awk '{print \$5}' | sed 's/%//'")
if [ $DISK_USAGE -gt 70 ]; then
    log "WARNING: Disk usage is ${DISK_USAGE}%"
    send_notification "VyOS Update WARNING" "High disk usage (${DISK_USAGE}%) on $ROUTER_IP"

    # Cleanup old images if needed
    log "Cleaning up old images"
    ssh vyos@$ROUTER_IP "delete system image oldest"
fi

# Backup configuration
log "Creating configuration backup"
BACKUP_FILE="$BACKUP_DIR/$(date +%Y%m%d-%H%M%S)-$ROUTER_IP.config"
ssh vyos@$ROUTER_IP "save /tmp/backup.config"
scp vyos@$ROUTER_IP:/tmp/backup.config $BACKUP_FILE

if [ ! -f $BACKUP_FILE ]; then
    log "ERROR: Backup failed"
    send_notification "VyOS Update FAILED" "Backup creation failed for $ROUTER_IP"
    exit 1
fi

log "Backup created: $BACKUP_FILE"

# Check for updates
log "Checking for available updates"
UPDATE_INFO=$(ssh vyos@$ROUTER_IP "show system updates" 2>&1)

if echo "$UPDATE_INFO" | grep -q "Update available"; then
    NEW_VERSION=$(echo "$UPDATE_INFO" | grep "Update available" | awk '{print $3}')
    log "Update available: $NEW_VERSION"

    # Download and install new image
    log "Installing new image: $NEW_VERSION"
    ssh vyos@$ROUTER_IP "add system image latest" < /dev/null

    # Set as default boot
    log "Setting new image as default boot"
    ssh vyos@$ROUTER_IP "set system image default-boot $NEW_VERSION"

    # Schedule reboot in 5 minutes (safety window for cancellation)
    log "Scheduling reboot in 5 minutes"
    ssh vyos@$ROUTER_IP "shutdown reboot +5 'Automatic update to $NEW_VERSION'"

    send_notification "VyOS Update SCHEDULED" "Router $ROUTER_IP will reboot in 5 minutes to apply update to $NEW_VERSION"

    # Wait for reboot to complete
    log "Waiting for reboot to complete"
    sleep 360  # 6 minutes

    # Post-reboot verification
    log "Starting post-reboot verification"
    RETRIES=10
    while [ $RETRIES -gt 0 ]; do
        if ssh -o ConnectTimeout=5 vyos@$ROUTER_IP "show version" > /dev/null 2>&1; then
            log "Router is back online"
            break
        fi
        log "Waiting for router to come back online (retries left: $RETRIES)"
        sleep 30
        ((RETRIES--))
    done

    if [ $RETRIES -eq 0 ]; then
        log "ERROR: Router did not come back online after reboot"
        send_notification "VyOS Update CRITICAL" "Router $ROUTER_IP did not come back online after update. Manual intervention required!"
        exit 1
    fi

    # Verify new version is running
    CURRENT_VERSION=$(ssh vyos@$ROUTER_IP "show version | grep Version | awk '{print \$2}'")
    if [ "$CURRENT_VERSION" == "$NEW_VERSION" ]; then
        log "SUCCESS: Update completed successfully to $NEW_VERSION"
        send_notification "VyOS Update SUCCESS" "Router $ROUTER_IP successfully updated to $NEW_VERSION"
    else
        log "WARNING: Version mismatch. Expected $NEW_VERSION, got $CURRENT_VERSION"
        send_notification "VyOS Update WARNING" "Version mismatch on $ROUTER_IP. Expected $NEW_VERSION, got $CURRENT_VERSION"
    fi

    # Run post-update checks
    log "Running post-update health checks"

    # Check interfaces
    if ! ssh vyos@$ROUTER_IP "show interfaces" | grep -q "eth0"; then
        log "ERROR: Interfaces not configured properly"
        send_notification "VyOS Update ISSUE" "Interfaces not configured on $ROUTER_IP after update"
    fi

    # Check routing
    if ! ssh vyos@$ROUTER_IP "show ip route" | grep -q "default"; then
        log "ERROR: Default route missing"
        send_notification "VyOS Update ISSUE" "Default route missing on $ROUTER_IP after update"
    fi

    log "Update process completed"
else
    log "No updates available"
fi

log "Update script finished"
```

### Scheduling automatic updates

```bash
# Configure cron on the management server for weekly updates
# /etc/cron.d/vyos-updates

# Update every Sunday at 3:00 AM
0 3 * * 0 /opt/scripts/auto-update-vyos.sh >> /var/log/vyos-auto-update.log 2>&1
```

## Useful commands

```bash
# Viewing the version
show version
show version brief

# Managing images
show system image
add system image <url|latest>
delete system image <name>
rename system image <old-name> to <new-name>
set system image default-boot <name>

# Checking for updates
show system updates
show configuration system update-check

# Backup
save <filename>
save /config/backup-$(date +%Y%m%d).config

# Reboot
reboot
reboot now
shutdown reboot +5
shutdown reboot 18:00

# Monitoring
show system storage
show system uptime
show log | match image
df -h

# Rollback
set system image default-boot <previous-version>
# via GRUB: select the previous image at boot
```

## Conclusion

Proper management of VyOS updates is critical for maintaining the security and stability of your network infrastructure. The image-based mechanism provides:

- Safe testing of new versions
- Fast rollback when problems occur
- Version isolation
- Preservation of the configuration between updates

**Key recommendations:**

1. **Always create a backup** before an update
2. **Test updates** in a lab environment
3. **Plan maintenance windows** for production systems
4. **Document** all changes
5. **Monitor the system** after an update
6. **Have a rollback plan** in case of problems
7. **Use staged rollout** for large infrastructures
8. **Verify the compatibility** of the configuration with the new version
9. **Follow the VyOS release notes** for known issues
10. **Automate** the process where possible, but with safety checks

For production environments, we recommend using LTS versions of VyOS and applying updates only after thorough testing in a staging environment. Rolling release versions are suitable for labs and development environments where access to the latest features is important.