Syslog and Logging in VyOS

Syslog is the standard protocol for transmitting log messages across IP networks. VyOS provides a full-featured logging system, including local log files, remote syslog servers, buffered logs, and filtering by severity level.

Overview

VyOS uses rsyslog as its system logging daemon. The syslog system allows you to:

  • Centrally collect logs from many devices
  • Separate messages by facility (source) and severity (importance level)
  • Send logs to remote servers for analysis and archiving
  • Configure log rotation and retention
  • Integrate with SIEM systems (Splunk, ELK, Graylog)

VyOS Logging Architecture

VyOS supports several logging destinations:

  • Local files: Logs are written to files in /var/log/
  • Console: Messages are printed to the physical console
  • Buffer: Logs are kept in memory (ring buffer)
  • Remote servers: Forwarding to external syslog servers
  • Users: Delivery of critical messages to logged-in users

Basic Configuration

Local Logging to a File

# Логи всех уровней в файл /var/log/messages
set system syslog global facility all level info

# Отдельные логи для различных facility
set system syslog file messages facility all level info
set system syslog file auth.log facility auth level info
set system syslog file auth.log facility authpriv level info

Console Logging

# Вывод критических сообщений на консоль
set system syslog console facility all level err

Buffer Logging

# Circular buffer размером 10000 сообщений
set system syslog global facility all level info
set system syslog global archive size 10000

Remote Syslog Server

# Отправка всех логов на удаленный сервер
set system syslog host 192.168.1.100 facility all level info

# Использование TCP вместо UDP
set system syslog host 192.168.1.100 facility all protocol tcp

# Нестандартный порт
set system syslog host 192.168.1.100 port 5140

Facility (Sources)

VyOS supports the standard syslog facilities:

FacilityDescriptionExamples
allAll sourcesGeneral logging
authAuthenticationSSH login, sudo
authprivPrivate authenticationPAM, local logins
cronTask schedulerCron jobs
daemonSystem daemonsServices (DHCP, DNS, NTP)
kernSystem kernelKernel messages
mailMail systemMail subsystem
syslogSyslog daemonRsyslog messages
userUser processesUser commands
local0-local7User-definedCustom applications

Severity Levels

Severity levels in descending order:

LevelNumericDescriptionUsage
emerg0System is unusableCritical outage
alert1Immediate action requiredSerious error
crit2Critical conditionsHardware failure
err3Error conditionsOperation errors
warning4Warning conditionsWarnings
notice5Normal but significantConfiguration changes
info6Informational messagesStandard operation
debug7Debug messagesDetailed information

When you set a level, all messages of that level and above are recorded. For example, level warning records warning, err, crit, alert, and emerg.

Advanced Configuration

Multiple Remote Servers

# Основной syslog-сервер (все логи)
set system syslog host 192.168.1.100 facility all level info

# Резервный сyslog-сервер
set system syslog host 192.168.1.101 facility all level info

# Специализированный сервер только для security events
set system syslog host 192.168.1.200 facility auth level info
set system syslog host 192.168.1.200 facility authpriv level info

Selective Logging

# Только критические сообщения ядра в отдельный файл
set system syslog file kernel.log facility kern level err

# Только логи аутентификации
set system syslog file auth.log facility auth level info
set system syslog file auth.log facility authpriv level info

# Логи DHCP-сервера
set system syslog file dhcpd.log facility daemon level info

Logging with TLS (Encrypted Syslog)

VyOS 1.5+ supports TLS for secure log transmission:

# Настройка TLS для удаленного сервера
set system syslog host 192.168.1.100 facility all level info
set system syslog host 192.168.1.100 protocol tcp
set system syslog host 192.168.1.100 port 6514

# Использование сертификата для проверки сервера
set pki ca ca-syslog certificate 'MIIDXTCCAkWg...'

Configuring Archiving and Rotation

# Архивирование с ротацией логов
set system syslog global archive size 10000
set system syslog global archive file 10

# Размер файла логов перед ротацией (в байтах)
set system syslog file messages archive size 10485760
set system syslog file messages archive file 5

Configuration Examples

Example 1: Basic Local Logging

# Общие логи
set system syslog file messages facility all level info
set system syslog file messages archive size 10485760
set system syslog file messages archive file 5

# Логи аутентификации отдельно
set system syslog file auth.log facility auth level info
set system syslog file auth.log facility authpriv level info

# Критические сообщения на консоль
set system syslog console facility all level err

# Commit и сохранение
commit
save

Example 2: Centralized Logging (Enterprise)

# Локальные логи (для быстрой диагностики)
set system syslog file messages facility all level info

# Основной удаленный syslog-сервер
set system syslog host 192.168.1.100 facility all level info

# Резервный syslog-сервер
set system syslog host 192.168.1.101 facility all level warning

# Отдельный сервер безопасности (только auth)
set system syslog host 192.168.10.50 facility auth level info
set system syslog host 192.168.10.50 facility authpriv level info

# Консоль только для критических сообщений
set system syslog console facility all level crit

commit
save

Example 3: High-Load Environment with TCP

# TCP для гарантированной доставки
set system syslog host 192.168.1.100 facility all level info
set system syslog host 192.168.1.100 protocol tcp
set system syslog host 192.168.1.100 port 514

# Буферизованное логирование
set system syslog global facility all level info
set system syslog global archive size 50000

# Локальное логирование минимизировано
set system syslog file messages facility all level warning

commit
save

Example 4: Debugging a Specific Service (DHCP)

# Детальное логирование DHCP
set system syslog file dhcpd.log facility daemon level debug
set system syslog host 192.168.1.100 facility daemon level debug

# Обычные логи для остальных сервисов
set system syslog file messages facility all level info

commit
save

# После отладки - вернуть уровень info
delete system syslog file dhcpd.log facility daemon level debug
set system syslog file dhcpd.log facility daemon level info
commit
save

Example 5: Integration with the ELK Stack (Elasticsearch/Logstash/Kibana)

# Отправка JSON-форматированных логов в Logstash
set system syslog host 192.168.1.150 facility all level info
set system syslog host 192.168.1.150 port 5140
set system syslog host 192.168.1.150 protocol tcp

# Локальные логи для резервирования
set system syslog file messages facility all level info

commit
save

Logstash configuration (on the server side):

input {
  syslog {
    port => 5140
    type => "vyos"
  }
}

filter {
  if [type] == "vyos" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:hostname} %{DATA:program}(?:\[%{POSINT:pid}\])?: %{GREEDYDATA:log_message}" }
    }
    date {
      match => [ "timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "vyos-logs-%{+YYYY.MM.dd}"
  }
}

Example 6: Integration with Splunk

# Отправка всех логов в Splunk через TCP
set system syslog host splunk.example.com facility all level info
set system syslog host splunk.example.com port 514
set system syslog host splunk.example.com protocol tcp

# Локальное резервирование
set system syslog file messages facility all level info

commit
save

Splunk configuration (inputs.conf):

[tcp://514]
connection_host = ip
sourcetype = syslog
index = network_devices

[syslog]
TRANSFORMS-routing = route_vyos_logs

# Создание поля для VyOS устройств
[vyos-sourcetype]
REGEX = vyos
FORMAT = sourcetype::vyos_syslog
DEST_KEY = MetaData:Sourcetype

Monitoring and Diagnostics

Viewing Local Logs

# Просмотр текущих логов в реальном времени
monitor log

# Последние 50 строк
show log tail 50

# Логи аутентификации
show log auth tail 50

# Логи ядра
show log kernel tail 50

# Фильтрация по ключевому слову
show log | match DHCP
show log | match failed

# Логи конкретного файла
tail -f /var/log/messages
tail -f /var/log/auth.log

Checking the Syslog Configuration

# Показать активную конфигурацию syslog
show configuration system syslog

# Показать в формате commands
show configuration commands | match syslog

Checking rsyslog Status

# Статус демона rsyslog
show system syslog

# Проверка процесса
ps aux | grep rsyslog

# Проверка сетевых соединений (если используются удаленные серверы)
netstat -an | grep :514
ss -tulpn | grep rsyslog

Testing Log Delivery

# Генерация тестового сообщения
logger -t test-message "This is a test syslog message from VyOS"

# С указанием priority
logger -p local0.info "Test message to local0 facility"

# Проверка доставки на удаленный сервер (с сервера логов)
tail -f /var/log/syslog | grep vyos

Checking Network Reachability of the Syslog Server

# Ping до syslog-сервера
ping 192.168.1.100 count 3

# Проверка порта UDP 514
nc -vzu 192.168.1.100 514

# Проверка порта TCP 514
nc -vz 192.168.1.100 514

# Telnet (для TCP)
telnet 192.168.1.100 514

Troubleshooting

Problem: Logs Are Not Sent to the Remote Server

Diagnostics:

# 1. Проверка конфигурации
show configuration system syslog host

# 2. Проверка сетевой связности
ping 192.168.1.100 count 5

# 3. Проверка firewall на VyOS
show firewall

# 4. Проверка открытого порта на сервере (если есть доступ)
nc -vzu 192.168.1.100 514  # UDP
nc -vz 192.168.1.100 514   # TCP

# 5. Проверка логов rsyslog
grep -i error /var/log/messages | grep rsyslog
journalctl -u rsyslog -n 50

Solution:

# Проверка правил firewall (разрешить исходящий трафик)
set firewall name WAN_LOCAL rule 100 action accept
set firewall name WAN_LOCAL rule 100 description 'Allow syslog to remote server'
set firewall name WAN_LOCAL rule 100 destination address 192.168.1.100
set firewall name WAN_LOCAL rule 100 destination port 514
set firewall name WAN_LOCAL rule 100 protocol udp

# Или использовать TCP
delete system syslog host 192.168.1.100 protocol
set system syslog host 192.168.1.100 protocol tcp

commit
save

Problem: Logs Are Too Large, the Disk Is Filling Up

Diagnostics:

# Проверка использования диска
show system storage

# Размер файлов логов
ls -lh /var/log/

# Самые большие лог-файлы
du -h /var/log/ | sort -rh | head -10

Solution:

# Настройка ротации и архивирования
set system syslog file messages archive size 5242880    # 5 MB
set system syslog file messages archive file 3          # Хранить 3 архива

# Уменьшение уровня логирования
delete system syslog file messages facility all level debug
set system syslog file messages facility all level warning

# Отправка на удаленный сервер вместо локального хранения
set system syslog host 192.168.1.100 facility all level info
delete system syslog file messages

commit
save

# Ручная очистка старых логов
sudo rm /var/log/messages.*.gz

Problem: Logs from a Specific Service Are Not Visible

Diagnostics:

# Проверка, что facility настроен
show configuration system syslog | match daemon

# Проверка работы сервиса (например DHCP)
show service dhcp-server statistics

# Поиск логов сервиса
grep -i dhcp /var/log/messages

Solution:

# Добавление специфичного facility и level debug
set system syslog file dhcpd.log facility daemon level debug

# Или повышение детализации для всех
set system syslog file messages facility all level debug

commit
save

# После отладки - вернуть info
set system syslog file messages facility all level info
commit
save

Problem: Timestamps in the Logs Are Incorrect

Diagnostics:

# Проверка системного времени
show date

# Проверка NTP
show ntp

# Проверка timezone
show configuration system time-zone

Solution:

# Установка правильной timezone
set system time-zone Europe/Moscow

# Настройка NTP (если не настроен)
set system ntp server 0.pool.ntp.org
set system ntp server 1.pool.ntp.org

commit
save

# Перезапуск rsyslog
sudo systemctl restart rsyslog

Problem: Duplicate Messages in the Logs

Cause: This often occurs due to an incorrect facility configuration.

Solution:

# Проверка дублирующих правил
show configuration system syslog

# Пример некорректной конфигурации (дублирование)
# set system syslog file messages facility all level info
# set system syslog file messages facility kern level info  # Дублирует, т.к. all включает kern

# Удаление избыточных правил
delete system syslog file messages facility kern

commit
save

Problem: Logs Stop Being Written After a Configuration Change

Solution:

# Перезапуск rsyslog
sudo systemctl restart rsyslog

# Проверка статуса
sudo systemctl status rsyslog

# Проверка синтаксиса конфигурации rsyslog
sudo rsyslogd -N1

SIEM Integration

Graylog

# Настройка отправки в Graylog (GELF UDP input)
set system syslog host graylog.example.com facility all level info
set system syslog host graylog.example.com port 12201

commit
save

Graylog configuration:

  1. System → Inputs → Select “Syslog UDP”
  2. Launch a new input on port 514 or 12201
  3. Create extractors to parse VyOS logs

Syslog-ng (on the Server Side)

source s_vyos {
    udp(ip(0.0.0.0) port(514));
    tcp(ip(0.0.0.0) port(514));
};

filter f_vyos {
    host("vyos-*");
};

destination d_vyos {
    file("/var/log/vyos/$HOST/$YEAR-$MONTH-$DAY.log"
        create_dirs(yes)
        owner("root")
        group("root")
        perm(0640)
    );
};

log {
    source(s_vyos);
    filter(f_vyos);
    destination(d_vyos);
};

Rsyslog (on the Server Side)

# /etc/rsyslog.d/10-vyos.conf

# UDP input
module(load="imudp")
input(type="imudp" port="514")

# TCP input
module(load="imtcp")
input(type="imtcp" port="514")

# Шаблон для VyOS логов
template(name="VyOSLogFormat" type="string"
         string="/var/log/vyos/%HOSTNAME%/%$YEAR%-%$MONTH%-%$DAY%.log")

# Правило для VyOS устройств (по hostname)
if $hostname startswith 'vyos-' then {
    action(type="omfile" dynaFile="VyOSLogFormat")
    stop
}

Best Practices

1. Centralized Logging

  • Always send logs to a centralized syslog server
  • Use a backup server for critical environments
  • Keep only minimal local logs (warning and above) for quick diagnostics

2. Choosing a Protocol

  • UDP (514): Faster, but with no delivery guarantee. Suitable for most cases
  • TCP (514/601): Guaranteed delivery, slower. Use for critical logs
  • TLS (6514): Encrypted transmission for sensitive information

3. Logging Levels

  • Production: info for regular services, warning for high-load ones
  • Development/Testing: debug for troubleshooting specific problems
  • Security critical: info for auth/authpriv to a dedicated server

4. Rotation and Archiving

  • Configure rotation for all local logs
  • File size: 5-10 MB for routers with limited storage
  • Number of archives: 3-5 for history

5. Log Monitoring

  • Set up alerts for critical messages (crit, alert, emerg)
  • Monitor availability of the syslog server
  • Watch the size of local logs (disk space)

6. Security

  • Restrict access to the syslog server with a firewall
  • Use TLS to transmit logs over untrusted networks
  • Regularly review authentication logs

7. Performance

  • In high-load environments, minimize local logging
  • Use buffering to reduce disk load
  • Consider using TCP for stability

8. Time and Synchronization

  • Always configure NTP for accurate timestamps
  • Use the same timezone on all devices
  • Verify time synchronization regularly

9. Documentation

  • Document the logging scheme (which logs go where)
  • Specify the facility for each type of message
  • Keep a change log of the syslog configuration

10. Testing

  • Test syslog configuration changes using logger
  • Verify log delivery after configuring
  • Have a recovery plan for centralized logging failures

Useful Commands

# Просмотр логов в реальном времени
monitor log

# Последние N строк
show log tail 100

# Поиск по ключевому слову
show log | match "keyword"

# Логи аутентификации
show log auth tail 50

# Логи ядра
show log kernel tail 50

# Генерация тестового сообщения
logger -t test "Test syslog message"

# Проверка статуса rsyslog
show system syslog

# Перезапуск rsyslog (если необходимо)
sudo systemctl restart rsyslog

# Проверка синтаксиса конфигурации rsyslog
sudo rsyslogd -N1

# Просмотр конфигурации
show configuration system syslog

# Список файлов логов
ls -lh /var/log/

# Использование диска
show system storage
df -h

# Поиск по всем логам
sudo grep -r "search term" /var/log/

Conclusion

Proper syslog configuration is critical for monitoring, auditing, and troubleshooting network infrastructure. VyOS provides a flexible logging system with support for centralized servers, multiple severity levels, and integration with modern SIEM systems.

Centralized logging allows you to:

  • Correlate events across many devices
  • Perform security analysis
  • Meet compliance requirements
  • Diagnose problems quickly

For production environments, we recommend using a centralized syslog server with redundancy, configuring rotation of local logs, and monitoring the availability of the logging system.

Reviewed by OpenNix LLC · Last updated on