# Wazuh Log Data Collection - Event Log Ingestion

> Log collection in Wazuh 4.14 - localfile, syslog, JSON, EventChannel, macOS ULS, multi-line logs, remote syslog, and log rotation handling

Source: https://opennix.org/en/docs/wazuh/capabilities/wazuh-log-data-collection/


The Logcollector module in Wazuh gathers and consolidates event logs from various sources - local files, Windows Event Log, macOS Unified Logging System, remote devices via syslog, and command output. The collected data is forwarded to the Wazuh server, where the Analysisd module performs decoding, rule matching, and alert generation. Proper log collection configuration forms the foundation of the entire security monitoring system.

## Log Source Overview

Wazuh supports data collection from the following source types:

- **File-based logs** - syslog, JSON, Apache, Nginx, PostgreSQL, MySQL, and other formats
- **Windows Event Log** - via the EventChannel mechanism with XPATH filtering support
- **macOS Unified Logging System** - native ULS integration using predicate filters
- **systemd journal** - collection from journald on Linux systems
- **Remote syslog** - receiving messages from network devices and applications
- **Command output** - periodic command execution and output analysis

## Localfile Configuration

The primary configuration element is the `<localfile>` block in the `ossec.conf` file on the agent. Each block defines a single data source.

### File Log Collection

```xml
<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/syslog</location>
</localfile>

<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/auth.log</location>
</localfile>
```

| Parameter | Description |
|-----------|-------------|
| `location` | Path to the log file, event channel, `macos`, or `journald` |
| `log_format` | Log format (determines the parsing method) |
| `only-future-events` | Read only new entries after startup (yes/no, default yes) |
| `ignore_binaries` | Skip binary files (yes/no) |
| `age` | Process only recently modified files (e.g., `1d`) |
| `exclude` | Pattern to exclude files from collection |

### Path Wildcards and Date Patterns

Wazuh supports wildcards and date patterns in the `location` parameter:

```xml
<!-- Wildcard pattern -->
<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/*.log</location>
</localfile>

<!-- Date pattern -->
<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/app-%Y-%m-%d.log</location>
</localfile>
```

## Log Format Types (log_format)

### syslog

Standard syslog message format. Used for most system logs on Linux and Unix systems.

```xml
<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/messages</location>
</localfile>
```

### json

JSON-structured logs. Wazuh automatically parses JSON object fields and makes them available to detection rules.

```xml
<localfile>
  <log_format>json</log_format>
  <location>/var/log/app/events.json</location>
  <label key="@source">application</label>
</localfile>
```

### multi-line

Logs where a single event spans multiple lines. The parameter specifies a fixed number of lines per event.

```xml
<localfile>
  <log_format>multi-line:3</log_format>
  <location>/var/log/multiline-app.log</location>
</localfile>
```

The number after the colon indicates how many lines are combined into a single event.

### multi-line-regex

For logs with a variable number of lines per event, a regular expression defines the start or end of each record:

```xml
<localfile>
  <log_format>multi-line-regex</log_format>
  <location>/var/log/java-app.log</location>
  <multiline_regex>^\d{4}-\d{2}-\d{2}</multiline_regex>
</localfile>
```

In this example, each new event starts with a date in `YYYY-MM-DD` format.

### eventchannel

Windows event collection via the Event Channel API. Supports XPATH filtering to select specific events.

```xml
<localfile>
  <location>Security</location>
  <log_format>eventchannel</log_format>
  <query>Event/System[EventID != 5145 and EventID != 5156]</query>
</localfile>
```

### macos

macOS log collection via the Unified Logging System (ULS). Supports predicate filters for refining the selection.

```xml
<localfile>
  <location>macos</location>
  <log_format>macos</log_format>
  <query type="log,trace" level="info">process == "sshd" OR process == "sudo"</query>
</localfile>
```

### journald

Collection from the systemd journal:

```xml
<localfile>
  <location>journald</location>
  <log_format>journald</log_format>
  <filter field="_SYSTEMD_UNIT">sshd.service</filter>
</localfile>
```

### command and full_command

Command execution and output analysis. `command` processes each line individually, while `full_command` treats the entire output as a single event.

```xml
<localfile>
  <log_format>command</log_format>
  <command>df -P</command>
  <frequency>360</frequency>
</localfile>

<localfile>
  <log_format>full_command</log_format>
  <command>netstat -tulnp</command>
  <frequency>120</frequency>
  <alias>netstat-listening</alias>
</localfile>
```

### audit

Format for reading the Linux Audit (auditd) log:

```xml
<localfile>
  <log_format>audit</log_format>
  <location>/var/log/audit/audit.log</location>
</localfile>
```

## Remote Syslog

To receive syslog messages from devices that do not support agent installation (routers, switches, firewalls), configure the `<remote>` block on the Wazuh server:

```xml
<remote>
  <connection>syslog</connection>
  <port>514</port>
  <protocol>udp</protocol>
  <allowed-ips>10.0.0.0/8</allowed-ips>
</remote>
```

| Parameter | Description |
|-----------|-------------|
| `connection` | Connection type: `syslog` or `secure` (for agents) |
| `port` | Listening port (default 514) |
| `protocol` | Transport protocol: `udp` or `tcp` |
| `allowed-ips` | Allowed IP addresses and subnets |
| `local_ip` | Local IP address to bind to |
| `ipv6` | Enable IPv6 support (yes/no) |

For TCP connections:

```xml
<remote>
  <connection>syslog</connection>
  <port>514</port>
  <protocol>tcp</protocol>
  <allowed-ips>192.168.1.0/24</allowed-ips>
</remote>
```

## Windows EventChannel

Windows Event Log monitoring uses the `eventchannel` format. All standard channels (Security, System, Application) and custom application channels are supported.

### Standard Channels

```xml
<localfile>
  <location>Security</location>
  <log_format>eventchannel</log_format>
</localfile>

<localfile>
  <location>System</location>
  <log_format>eventchannel</log_format>
</localfile>

<localfile>
  <location>Application</location>
  <log_format>eventchannel</log_format>
</localfile>
```

### XPATH Filtering

XPATH queries allow selecting specific events:

```xml
<!-- Only logon events -->
<localfile>
  <location>Security</location>
  <log_format>eventchannel</log_format>
  <query>Event/System[EventID=4624 or EventID=4625]</query>
</localfile>

<!-- Exclude noisy events -->
<localfile>
  <location>Security</location>
  <log_format>eventchannel</log_format>
  <query>Event/System[EventID != 5145 and EventID != 5156 and EventID != 4658]</query>
</localfile>
```

### Sysmon Channels

```xml
<localfile>
  <location>Microsoft-Windows-Sysmon/Operational</location>
  <log_format>eventchannel</log_format>
</localfile>
```

### PowerShell Channels

```xml
<localfile>
  <location>Microsoft-Windows-PowerShell/Operational</location>
  <log_format>eventchannel</log_format>
</localfile>
```

## macOS Unified Logging System

On macOS, the Wazuh agent uses the Unified Logging System for collecting system and application logs. Filtering is performed through predicate expressions.

```xml
<localfile>
  <location>macos</location>
  <log_format>macos</log_format>
  <query type="log,trace" level="info">
    process == "sshd" OR process == "sudo" OR process == "login"
  </query>
</localfile>
```

| Query attribute | Description |
|-----------------|-------------|
| `type` | Record types: `log`, `trace`, `activity` |
| `level` | Minimum level: `default`, `info`, `debug` |

Predicate filter examples:

```xml
<!-- Authorization monitoring -->
<localfile>
  <location>macos</location>
  <log_format>macos</log_format>
  <query type="log" level="info">subsystem == "com.apple.Authorization"</query>
</localfile>

<!-- Network connection monitoring -->
<localfile>
  <location>macos</location>
  <log_format>macos</log_format>
  <query type="log" level="default">category == "connection"</query>
</localfile>
```

## Labels and Tags

Labels add custom fields to events, simplifying categorization and search:

```xml
<localfile>
  <log_format>json</log_format>
  <location>/var/log/webapp/access.json</location>
  <label key="@source">webapp-frontend</label>
  <label key="@environment">production</label>
</localfile>
```

Labels are included in the alert JSON output and are available for filtering in the Wazuh Dashboard.

## Log Rotation Handling

Wazuh automatically tracks log file rotation. The Logcollector module detects when a file has been recreated or truncated and starts reading from the beginning of the new file. For files with dates in their names, use date patterns:

```xml
<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/app/app-%Y-%m-%d.log</location>
</localfile>
```

The `age` parameter limits processing to recent files only:

```xml
<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/archive/*.log</location>
  <age>7d</age>
</localfile>
```

This example processes only files modified within the last 7 days.

## Output Redirection (target and out_format)

Wazuh supports redirecting logs to additional sockets and formatting the output:

```xml
<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/secure</location>
  <target>agent,custom_socket</target>
  <out_format target="custom_socket">
    %(hostname)s: %(log)s
  </out_format>
</localfile>
```

## The restrict and ignore Parameters

Line-level filtering with regular expressions:

```xml
<!-- Process only error lines -->
<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/application.log</location>
  <restrict>ERROR|CRITICAL|FATAL</restrict>
</localfile>

<!-- Skip debug lines -->
<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/application.log</location>
  <ignore>DEBUG|TRACE</ignore>
</localfile>
```

## Comparison with Other Platforms

| Capability | Wazuh Logcollector | Splunk Inputs | Logstash / Filebeat |
|------------|-------------------|---------------|---------------------|
| File logs | localfile with patterns | monitor / inputs.conf | file input plugin / filebeat |
| Windows EventLog | eventchannel + XPATH | WinEventLog | winlogbeat |
| macOS ULS | Native support | No native support | No native support |
| Remote syslog | Built-in receiver | Built-in (UDP/TCP) | syslog input plugin |
| JSON logs | Automatic parsing | Automatic parsing | json codec |
| Multi-line | Fixed and regex modes | Configurable | multiline codec |
| Commands | command / full_command | scripted input | exec input |
| Agent-side filtering | restrict / ignore | No (server-side filtering) | processors / include_lines |
| Cost | Free (open source) | Commercial | Free (basic) |

## Log Processing Pipeline

Collected logs pass through three processing stages on the Wazuh server:

1. **Pre-decoding** - extraction of basic fields (timestamp, hostname, program)
2. **Decoding** - structured field parsing using decoders
3. **Rule matching** - alert generation based on detection rules

All logs (including those that do not trigger alerts) are saved to the `/var/ossec/logs/archives/` archive for retrospective analysis, provided archiving is enabled.

More about decoders: [Wazuh Architecture](/docs/wazuh/getting-started/wazuh-architecture/)

## Troubleshooting

### Logs Are Not Collected

- Verify the path in `<location>` exists and is readable by the `wazuh` user
- Confirm `<log_format>` matches the actual log format
- Check `/var/ossec/logs/ossec.log` for Logcollector errors
- For wildcard patterns, ensure files match the specified pattern

### Windows EventChannel Not Working

- Verify the exact event channel name (case-sensitive)
- Validate the XPATH query in `<query>` through Windows Event Viewer
- Confirm `<reconnect_time>` is set (default 5s)
- Ensure the agent runs with administrator privileges

### macOS ULS Not Returning Events

- Check the predicate expression syntax in `<query>`
- Verify the level is not set too high for the target events
- Test the predicate using the `log stream --predicate` command in the terminal

### High Resource Consumption

- Use `<restrict>` and `<ignore>` to filter unnecessary lines
- Increase `<frequency>` for commands with frequent output
- Limit `<age>` for directories with many files
- Apply XPATH filtering for EventChannel to exclude noisy events

More about agent installation: [Installing the Wazuh Agent](/docs/wazuh/installation/wazuh-agent-installation/)

More about use cases: [Wazuh Use Cases](/docs/wazuh/getting-started/wazuh-use-cases/)