Configure syslog monitor for multiple log files

Introduction

The Scalyr Agent can be configured to aggregate syslog events with its built-in syslog monitor. When the syslog monitor is employed, log events are directed by default to the agent_syslog.log file and the agentSyslog parser. Although this works well for a standalone log format, if multiple log formats are issued to syslog by this method (ex. firewall logs, system service logs (like DHCP, NTP, and cron), and application logs), the "agentSyslog" parser will become increasingly complex (and less maintainable) with each additional format.

Furthermore, open ended key-value pair parsing statements can't be used safely, as the likelihood of extracting incorrect data increases with overlapping formats.

Recommended Configuration

The following Scalyr Agent configuration can be used for when the syslog monitor receives logs from multiple sources. This configuration will support around 2.5MB/sec (or ~216GB/day) per Agent instance.

1. Multiple syslog monitors can be run per instance of the Scalyr Agent. Each monitor will listen for incoming logs on a different port.
2. Each entry assigns a unique log file (via the message_log field) and parser (using the parser field) to the log data

For example,

...
monitors: [
    {
      module: "scalyr_agent.builtin_monitors.syslog_monitor",
      protocols: "tcp:601, udp:514",
      accept_remote_connections: true,
      message_log: "firewall.log", 
      parser: "firewall"      
    },
    {
      module: "scalyr_agent.builtin_monitors.syslog_monitor",
      protocols: "tcp:602, udp:515",
      accept_remote_connections: true,
      message_log: "system.log", //send ntpdate, chrond, and dhcpd logs here
      parser: "system"      
    },
    {
      module: "scalyr_agent.builtin_monitors.syslog_monitor",
      protocols: "tcp:603, udp:516",
      accept_remote_connections: true,
      message_log: "sonicwall.log", //send SonicWall logs here
      parser: "sonicwall"      
    },
    ...
],
...

Further Reading

More information on the Scalyr Agent's built in syslog monitor can be found here: https://app.scalyr.com/monitors/syslog-monitor