Event Handler
Event Handler lets you run scripts in response to events in the VyOS system logs, providing automated reactions to system events.
Overview
Event Handler provides:
- Real-time monitoring of journald logs
- Execution of custom scripts when patterns match
- Passing environment variables to scripts
- Filtering by syslog identifiers
- Automated responses to system events
Event Handler works with journald (the systemd journal) and supports regular expressions for matching events.
Basic Configuration
Minimal Setup
A minimal Event Handler configuration:
set service event-handler event <event-name> filter pattern '<regex>'
set service event-handler event <event-name> script path '/config/scripts/handler.sh'
commitWhere:
<event-name>- the name of the event handler<regex>- the regular expression to match against logs/config/scripts/handler.sh- path to the executable script
Configuration Structure
service event-handler
└── event <event-name>
├── filter
│ ├── pattern <regex>
│ └── syslog-identifier <identifier>
└── script
├── path <script-path>
├── environment <name> value <value>
└── arguments <arguments>Event Handler Configuration
Creating an Event Handler
Create a new event handler:
set service event-handler event MY_EVENTDefining a Pattern
The regular expression to match against logs (a required parameter):
set service event-handler event MY_EVENT filter pattern 'interface eth0.*up'Important: The regular expression must match the entire message string, not just a part of it.
Pattern examples:
# Match for an interface
set service event-handler event LINK_DOWN filter pattern '.*link is no longer valid.*'
# Match for OSPF
set service event-handler event OSPF_CHANGE filter pattern '.*OSPF.*neighbor.*changed.*'
# Match for BGP
set service event-handler event BGP_DOWN filter pattern '.*BGP.*peer.*down.*'
# Match for VPN
set service event-handler event IPSEC_UP filter pattern '.*IPsec SA established.*'Specifying the Script
The path to the executable script:
set service event-handler event MY_EVENT script path '/config/scripts/my-handler.sh'Important: The script must be executable:
chmod +x /config/scripts/my-handler.shThe script can be written in any language:
- Shell (bash, sh)
- Python
- Perl
- Ruby
- Any other available interpreter
Filtering by Syslog Identifier
Restrict matching to a specific syslog identifier:
set service event-handler event MY_EVENT filter syslog-identifier 'kernel'Examples of syslog identifiers:
kernel- kernel eventssystemd- systemd eventssshd- SSH eventsvyos-configd- VyOS configuration eventsfrr- FRR (routing) eventschrony- NTP eventssnmpd- SNMP events
List active identifiers:
journalctl -F SYSLOG_IDENTIFIEREnvironment Variables
Pass environment variables to the script (the recommended method):
set service event-handler event MY_EVENT script environment VAR_NAME value 'value'Multiple variables:
set service event-handler event IF_DOWN script environment INTERFACE value 'eth0'
set service event-handler event IF_DOWN script environment ACTION value 'notify'
set service event-handler event IF_DOWN script environment EMAIL value 'admin@example.com'Built-in environment variable:
message- the full message that triggered the handler
Script Arguments
Pass arguments to the script (a legacy method, not recommended):
set service event-handler event MY_EVENT script arguments 'arg1 arg2 arg3'Recommendation: Use environment variables instead of arguments.
Script Examples
Shell Script - Basic Example
/config/scripts/interface-handler.sh:
#!/bin/bash
# Get the message
MESSAGE="${message}"
# Get custom variables
INTERFACE="${INTERFACE:-unknown}"
ACTION="${ACTION:-unknown}"
# Logging
logger -t event-handler "Event triggered: $ACTION on $INTERFACE"
logger -t event-handler "Message: $MESSAGE"
# Actions
case "$ACTION" in
down)
logger -t event-handler "Interface $INTERFACE is down!"
# Send a notification, restart the interface, etc.
;;
up)
logger -t event-handler "Interface $INTERFACE is up"
;;
esacPython Script - Interface Monitoring
/config/scripts/interface-monitor.py:
#!/usr/bin/env python3
import os
import sys
import logging
from datetime import datetime
# Logging configuration
logging.basicConfig(
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
level=logging.INFO
)
logger = logging.getLogger('event-handler')
def main():
# Get environment variables
message = os.environ.get('message', '')
interface = os.environ.get('interface_name', 'unknown')
action = os.environ.get('interface_action', 'unknown')
logger.info(f"Event triggered for interface: {interface}")
logger.info(f"Action: {action}")
logger.info(f"Full message: {message}")
# Handling logic
if action == 'down':
logger.warning(f"Interface {interface} is DOWN!")
# Send an alert, perform actions
send_alert(interface, action, message)
elif action == 'up':
logger.info(f"Interface {interface} is UP")
send_notification(interface, action)
def send_alert(interface, action, message):
"""Send a critical alert"""
# This could integrate with:
# - Email
# - Telegram
# - Slack
# - Syslog server
# - Monitoring system
pass
def send_notification(interface, action):
"""Send an informational notification"""
pass
if __name__ == '__main__':
try:
main()
except Exception as e:
logger.error(f"Error in event handler: {e}")
sys.exit(1)Python Script - Sending to Telegram
/config/scripts/telegram-alert.py:
#!/usr/bin/env python3
import os
import requests
import json
from datetime import datetime
# Telegram configuration (can be passed via environment)
TELEGRAM_BOT_TOKEN = os.environ.get('TELEGRAM_BOT_TOKEN', '')
TELEGRAM_CHAT_ID = os.environ.get('TELEGRAM_CHAT_ID', '')
def send_telegram_message(message):
"""Send a message to Telegram"""
if not TELEGRAM_BOT_TOKEN or not TELEGRAM_CHAT_ID:
print("Telegram credentials not configured")
return False
url = f"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendMessage"
payload = {
'chat_id': TELEGRAM_CHAT_ID,
'text': message,
'parse_mode': 'Markdown'
}
try:
response = requests.post(url, json=payload, timeout=10)
return response.status_code == 200
except Exception as e:
print(f"Failed to send Telegram message: {e}")
return False
def main():
# Get event information
log_message = os.environ.get('message', 'No message')
event_type = os.environ.get('event_type', 'unknown')
severity = os.environ.get('severity', 'info')
hostname = os.environ.get('hostname', 'vyos')
# Build the message
timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
telegram_text = f"""
*VyOS Event Alert*
*Hostname:* `{hostname}`
*Timestamp:* `{timestamp}`
*Event Type:* `{event_type}`
*Severity:* `{severity}`
*Message:*{log_message}
"""
# Send
if send_telegram_message(telegram_text):
print("Alert sent to Telegram successfully")
else:
print("Failed to send alert to Telegram")
if __name__ == '__main__':
main()Shell Script - Webhook Notification
/config/scripts/webhook-notify.sh:
#!/bin/bash
# Configuration
WEBHOOK_URL="${WEBHOOK_URL:-http://monitoring.example.com/webhook}"
MESSAGE="${message:-No message}"
EVENT_NAME="${event_name:-unknown}"
HOSTNAME=$(hostname)
# Build the JSON payload
JSON_PAYLOAD=$(cat <<EOF
{
"hostname": "$HOSTNAME",
"event": "$EVENT_NAME",
"message": "$MESSAGE",
"timestamp": "$(date -Iseconds)"
}
EOF
)
# Send the webhook
curl -X POST \
-H "Content-Type: application/json" \
-d "$JSON_PAYLOAD" \
"$WEBHOOK_URL" \
--max-time 10 \
--silent \
--show-error
exit 0Configuration Examples
Interface Monitoring
Track the interface state and run a script when it changes:
# Handler for an interface going down
set service event-handler event INTERFACE_DOWN filter pattern '.*eth0.*link is no longer valid.*'
set service event-handler event INTERFACE_DOWN filter syslog-identifier 'kernel'
set service event-handler event INTERFACE_DOWN script path '/config/scripts/interface-down.sh'
set service event-handler event INTERFACE_DOWN script environment INTERFACE value 'eth0'
set service event-handler event INTERFACE_DOWN script environment ACTION value 'down'
# Handler for an interface coming up
set service event-handler event INTERFACE_UP filter pattern '.*eth0.*link is now usable.*'
set service event-handler event INTERFACE_UP filter syslog-identifier 'kernel'
set service event-handler event INTERFACE_UP script path '/config/scripts/interface-up.sh'
set service event-handler event INTERFACE_UP script environment INTERFACE value 'eth0'
set service event-handler event INTERFACE_UP script environment ACTION value 'up'
commitBGP Neighbor Monitoring
# BGP neighbor down
set service event-handler event BGP_NEIGHBOR_DOWN filter pattern '.*BGP.*neighbor.*Down.*'
set service event-handler event BGP_NEIGHBOR_DOWN filter syslog-identifier 'bgpd'
set service event-handler event BGP_NEIGHBOR_DOWN script path '/config/scripts/bgp-alert.py'
set service event-handler event BGP_NEIGHBOR_DOWN script environment EVENT_TYPE value 'bgp_down'
set service event-handler event BGP_NEIGHBOR_DOWN script environment SEVERITY value 'critical'
# BGP neighbor up
set service event-handler event BGP_NEIGHBOR_UP filter pattern '.*BGP.*neighbor.*Up.*'
set service event-handler event BGP_NEIGHBOR_UP filter syslog-identifier 'bgpd'
set service event-handler event BGP_NEIGHBOR_UP script path '/config/scripts/bgp-alert.py'
set service event-handler event BGP_NEIGHBOR_UP script environment EVENT_TYPE value 'bgp_up'
set service event-handler event BGP_NEIGHBOR_UP script environment SEVERITY value 'info'
commitVPN Connection Monitoring
# IPsec tunnel established
set service event-handler event IPSEC_UP filter pattern '.*IPsec SA established.*'
set service event-handler event IPSEC_UP filter syslog-identifier 'charon'
set service event-handler event IPSEC_UP script path '/config/scripts/vpn-notify.sh'
set service event-handler event IPSEC_UP script environment VPN_STATUS value 'up'
# IPsec tunnel torn down
set service event-handler event IPSEC_DOWN filter pattern '.*IPsec SA deleted.*'
set service event-handler event IPSEC_DOWN filter syslog-identifier 'charon'
set service event-handler event IPSEC_DOWN script path '/config/scripts/vpn-notify.sh'
set service event-handler event IPSEC_DOWN script environment VPN_STATUS value 'down'
set service event-handler event IPSEC_DOWN script environment SEVERITY value 'warning'
commitSSH Login Monitoring
# Successful SSH login
set service event-handler event SSH_LOGIN filter pattern '.*Accepted publickey for.*'
set service event-handler event SSH_LOGIN filter syslog-identifier 'sshd'
set service event-handler event SSH_LOGIN script path '/config/scripts/ssh-login-notify.sh'
set service event-handler event SSH_LOGIN script environment EVENT_TYPE value 'ssh_login'
# Failed SSH attempt
set service event-handler event SSH_FAILED filter pattern '.*Failed password for.*'
set service event-handler event SSH_FAILED filter syslog-identifier 'sshd'
set service event-handler event SSH_FAILED script path '/config/scripts/ssh-failed-notify.sh'
set service event-handler event SSH_FAILED script environment EVENT_TYPE value 'ssh_failed'
set service event-handler event SSH_FAILED script environment SEVERITY value 'warning'
commitDisk Usage Monitoring
set service event-handler event DISK_FULL filter pattern '.*No space left on device.*'
set service event-handler event DISK_FULL script path '/config/scripts/disk-full-alert.py'
set service event-handler event DISK_FULL script environment ALERT_TYPE value 'disk_full'
set service event-handler event DISK_FULL script environment SEVERITY value 'critical'
set service event-handler event DISK_FULL script environment WEBHOOK_URL value 'https://monitoring.example.com/alerts'
commitTelegram Integration
set service event-handler event CRITICAL_ALERT filter pattern '.*(critical|panic|emergency).*'
set service event-handler event CRITICAL_ALERT script path '/config/scripts/telegram-alert.py'
set service event-handler event CRITICAL_ALERT script environment TELEGRAM_BOT_TOKEN value 'YOUR_BOT_TOKEN'
set service event-handler event CRITICAL_ALERT script environment TELEGRAM_CHAT_ID value 'YOUR_CHAT_ID'
set service event-handler event CRITICAL_ALERT script environment event_type value 'critical'
set service event-handler event CRITICAL_ALERT script environment severity value 'critical'
set service event-handler event CRITICAL_ALERT script environment hostname value 'vyos-gateway'
commitTemperature Monitoring
set service event-handler event HIGH_TEMP filter pattern '.*temperature.*critical.*'
set service event-handler event HIGH_TEMP filter syslog-identifier 'kernel'
set service event-handler event HIGH_TEMP script path '/config/scripts/temp-alert.sh'
set service event-handler event HIGH_TEMP script environment ALERT_TYPE value 'temperature'
set service event-handler event HIGH_TEMP script environment SEVERITY value 'critical'
commitIntegrations
Email Notifications
Set up email delivery through an external SMTP server:
/config/scripts/email-notify.sh:
#!/bin/bash
MESSAGE="${message}"
RECIPIENT="${EMAIL_TO:-admin@example.com}"
SUBJECT="${EMAIL_SUBJECT:-VyOS Event Alert}"
# Via the mail command (requires postfix/sendmail to be configured)
echo "$MESSAGE" | mail -s "$SUBJECT" "$RECIPIENT"
# Or via curl and SMTP
# curl --url 'smtp://mail.example.com:587' \
# --mail-from 'vyos@example.com' \
# --mail-rcpt "$RECIPIENT" \
# --user 'username:password' \
# --upload-file <(echo -e "Subject: $SUBJECT\n\n$MESSAGE")Configuration:
set service event-handler event EMAIL_ALERT script environment EMAIL_TO value 'admin@example.com'
set service event-handler event EMAIL_ALERT script environment EMAIL_SUBJECT value 'VyOS Critical Alert'Slack Webhook
/config/scripts/slack-notify.sh:
#!/bin/bash
SLACK_WEBHOOK="${SLACK_WEBHOOK_URL}"
MESSAGE="${message}"
HOSTNAME=$(hostname)
if [ -z "$SLACK_WEBHOOK" ]; then
echo "Slack webhook URL not configured"
exit 1
fi
# Build the JSON for Slack
JSON_PAYLOAD=$(cat <<EOF
{
"text": "VyOS Event on $HOSTNAME",
"attachments": [
{
"color": "danger",
"fields": [
{
"title": "Message",
"value": "$MESSAGE",
"short": false
}
]
}
]
}
EOF
)
# Send to Slack
curl -X POST \
-H 'Content-Type: application/json' \
-d "$JSON_PAYLOAD" \
"$SLACK_WEBHOOK"Configuration:
set service event-handler event SLACK_ALERT script environment SLACK_WEBHOOK_URL value 'https://hooks.slack.com/services/YOUR/WEBHOOK/URL'Syslog Server
Forward events to an external syslog server:
/config/scripts/syslog-forward.sh:
#!/bin/bash
SYSLOG_SERVER="${SYSLOG_SERVER:-192.168.1.100}"
SYSLOG_PORT="${SYSLOG_PORT:-514}"
MESSAGE="${message}"
FACILITY="${SYSLOG_FACILITY:-local0}"
SEVERITY="${SYSLOG_SEVERITY:-warning}"
# Send via logger to a remote syslog
logger -n "$SYSLOG_SERVER" \
-P "$SYSLOG_PORT" \
-p "${FACILITY}.${SEVERITY}" \
-t "vyos-event-handler" \
"$MESSAGE"Prometheus Alertmanager
/config/scripts/alertmanager.py:
#!/usr/bin/env python3
import os
import requests
import json
from datetime import datetime
ALERTMANAGER_URL = os.environ.get('ALERTMANAGER_URL', 'http://alertmanager:9093')
def send_alert():
message = os.environ.get('message', '')
alert_name = os.environ.get('alert_name', 'VyOSEvent')
severity = os.environ.get('severity', 'warning')
alert = [{
'labels': {
'alertname': alert_name,
'severity': severity,
'instance': os.uname().nodename,
},
'annotations': {
'summary': 'VyOS Event Triggered',
'description': message,
},
'startsAt': datetime.utcnow().isoformat() + 'Z',
}]
try:
response = requests.post(
f"{ALERTMANAGER_URL}/api/v1/alerts",
json=alert,
timeout=5
)
print(f"Alert sent: {response.status_code}")
except Exception as e:
print(f"Failed to send alert: {e}")
if __name__ == '__main__':
send_alert()Operational Commands
Viewing the Configuration
View all event handlers:
show service event-handlerA specific handler:
show service event-handler event MY_EVENTTesting Regular Expressions
Check whether a pattern matches the logs:
# View the last journal entries
journalctl -n 100 --no-pager
# Filter by identifier
journalctl -t kernel -n 50
# Search for a specific pattern
journalctl | grep -E 'your-regex-pattern'Debugging Scripts
Run the script manually with environment variables:
# Set environment variables
export message="Test event message"
export INTERFACE="eth0"
export ACTION="down"
# Run the script
/config/scripts/interface-handler.shViewing Execution Logs
Event handler logs:
journalctl -t vyos-event-handler
# In real time
journalctl -t vyos-event-handler -fLogs for a specific script (if it uses logger):
journalctl -t event-handler
# Search for errors
journalctl -t event-handler | grep -i errorRestarting the Service
restart event-handlerDisabling a Handler
Temporarily disable it without removing the configuration:
set service event-handler event MY_EVENT disable
commitRe-enable it:
delete service event-handler event MY_EVENT disable
commitTroubleshooting
The Script Does Not Run
- Check the execute permissions:
ls -l /config/scripts/my-script.sh
chmod +x /config/scripts/my-script.sh- Check the shebang in the script:
#!/bin/bash- Check the logs:
journalctl -t vyos-event-handler | tail -50The Pattern Does Not Trigger
- Check the journald logs:
journalctl -n 100 | grep -i "your-pattern"- Test the regular expression:
echo "test message" | grep -E 'your-regex'- Make sure the pattern matches the entire string:
# Wrong (partial match)
pattern 'eth0'
# Correct (full match)
pattern '.*eth0.*'The Script Runs but Does Not Work
- Check the environment variables in the script:
#!/bin/bash
env > /tmp/event-handler-env.log
echo "Message: ${message}" >> /tmp/event-handler-debug.log- Check the script output:
journalctl -t event-handler- Add debug output:
import logging
logging.basicConfig(level=logging.DEBUG)Problems with syslog-identifier
Check the available identifiers:
journalctl -F SYSLOG_IDENTIFIERExamples of real events:
journalctl -t kernel -n 20
journalctl -t bgpd -n 20
journalctl -t sshd -n 20The Script Does Not Receive Variables
Check the configuration:
show service event-handler event MY_EVENTAdd output of all variables to the script:
#!/bin/bash
env | grep -v PASSWORD >> /tmp/env-vars.logSecurity and Best Practices
1. Script Security
Set the correct permissions:
chown root:vyattacfg /config/scripts/handler.sh
chmod 750 /config/scripts/handler.shAvoid storing passwords in scripts:
# Bad
PASSWORD="secret123"
# Good
PASSWORD="${PASSWORD_ENV_VAR}"2. Limiting Execution
Add timeouts to scripts:
timeout 10 curl https://api.example.com/webhookUse locks to prevent concurrent execution:
LOCK_FILE="/tmp/event-handler.lock"
if [ -e "$LOCK_FILE" ]; then
echo "Already running"
exit 0
fi
touch "$LOCK_FILE"
trap "rm -f $LOCK_FILE" EXIT3. Error Handling
Always handle errors in scripts:
#!/bin/bash
set -e # Exit on error
set -u # Error on the use of undefined variables
set -o pipefail # Error in a pipe
# Error handler
trap 'echo "Error at line $LINENO"' ERRPython:
try:
# Your code
pass
except Exception as e:
logging.error(f"Error: {e}")
sys.exit(1)4. Logging
Always log execution:
logger -t event-handler "Starting handler for event: $EVENT_NAME"
logger -t event-handler "Completed successfully"5. Testing Patterns
Test your regex thoroughly before applying it:
# Save the log to a file
journalctl -n 1000 > /tmp/test.log
# Test the pattern
grep -E 'your-regex' /tmp/test.log6. Performance
Optimize your scripts:
- Use lightweight commands
- Avoid long-running operations
- Use background execution for long tasks:
long_running_task &7. Monitoring the Handlers Themselves
Create a watchdog to monitor the event handlers:
# Check that event-handler is running
systemctl status vyos-event-handler.service8. Documentation
Document each event handler:
set service event-handler event MY_EVENT description 'Monitors eth0 interface state and sends alerts'9. Script Versioning
Keep your scripts in git:
cd /config/scripts
git init
git add .
git commit -m "Initial event handler scripts"10. Backups
Save the configuration regularly:
saveBack up the scripts:
tar -czf /config/scripts-backup-$(date +%Y%m%d).tar.gz /config/scripts/Usage Examples
Automatic Interface Recovery
A script that automatically restarts an interface when it goes down:
/config/scripts/auto-recover-interface.sh:
#!/bin/bash
INTERFACE="${INTERFACE:-eth0}"
MAX_RETRIES=3
RETRY_COUNT=0
logger -t event-handler "Interface $INTERFACE went down, attempting recovery"
while [ $RETRY_COUNT -lt $MAX_RETRIES ]; do
sleep 5
# Attempt to bring the interface up
vtysh -c "configure terminal" -c "interface $INTERFACE" -c "no shutdown"
sleep 5
# Check the status
if ip link show "$INTERFACE" | grep -q "state UP"; then
logger -t event-handler "Interface $INTERFACE recovered successfully"
exit 0
fi
RETRY_COUNT=$((RETRY_COUNT + 1))
logger -t event-handler "Recovery attempt $RETRY_COUNT failed"
done
logger -t event-handler "Failed to recover interface $INTERFACE after $MAX_RETRIES attempts"
exit 1Dynamically Blocking an IP During an Attack
/config/scripts/block-attack.sh:
#!/bin/bash
MESSAGE="${message}"
# Extract the IP address from the message (example for SSH)
ATTACKER_IP=$(echo "$MESSAGE" | grep -oE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | head -1)
if [ -z "$ATTACKER_IP" ]; then
logger -t event-handler "Could not extract IP from message"
exit 1
fi
logger -t event-handler "Blocking attacker IP: $ATTACKER_IP"
# Add to the firewall address-group
vbash -c "configure" \
-c "set firewall group address-group BLOCKED_IPS address $ATTACKER_IP" \
-c "commit" \
-c "save"
logger -t event-handler "Successfully blocked $ATTACKER_IP"Automatically Updating DNS When the IP Changes
/config/scripts/update-dynamic-dns.py:
#!/usr/bin/env python3
import os
import re
import requests
def get_current_ip(message):
"""Extract the IP address from the message"""
ip_pattern = r'\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b'
match = re.search(ip_pattern, message)
return match.group(0) if match else None
def update_dns(ip_address):
"""Update the DNS record via the API"""
dns_provider_url = os.environ.get('DNS_API_URL')
api_key = os.environ.get('DNS_API_KEY')
hostname = os.environ.get('HOSTNAME')
if not all([dns_provider_url, api_key, hostname]):
print("DNS update configuration incomplete")
return False
try:
response = requests.post(
dns_provider_url,
json={
'hostname': hostname,
'ip': ip_address,
},
headers={'Authorization': f'Bearer {api_key}'},
timeout=10
)
return response.status_code == 200
except Exception as e:
print(f"Failed to update DNS: {e}")
return False
def main():
message = os.environ.get('message', '')
ip = get_current_ip(message)
if ip:
print(f"Detected new IP: {ip}")
if update_dns(ip):
print("DNS updated successfully")
else:
print("Failed to update DNS")
else:
print("Could not extract IP address")
if __name__ == '__main__':
main()Next Steps
- SNMP - monitoring via SNMP for integration with monitoring systems
- Syslog - configuring centralized logging
- Scripting Guide - a detailed guide to writing scripts for VyOS