Sending Palo Alto Networks Firewall Logs to DataSet

As part of the SentinelOne family, it has become increasingly important for DataSet to support various IT/security offerings. While we're still in the early phase of research and development, DataSet does have the ability to consume Syslogs today which is conveniently supported by many of today's firewalls and VPNs. In this article, I'd like to cover the instruction of sending Palo Alto Networks firewall traffic logs to Scalyr. 

1. Deploying the Scalyr Agent

Spinning up a server and installing the Scalyr agent. This server is served as a proxy Syslog server that accepts PAN firewall logs via Syslog protocol and redirects the data to DataSet. Since this server is only responsible for inbound Syslog traffic, the agent config requires nothing but the Syslog monitor configuration.

...
monitors: [
         {
             module:                    "scalyr_agent.builtin_monitors.syslog_monitor",
             protocols:                 "tcp:601, udp:514",
             accept_remote_connections: true,
             message_log: "syslog1.log",
             parser: "syslog-parser"
         }

   ]
...

In the above sample config, the agent accepts both TCP and UDP traffic from ports 601 and 514 respectively. The ingested data will be processed by the parser syslog-parser with the log file name "syslog1.log". 

Please refer to our Syslog documentation for additional information. 

2. Configuring Parser to Process PAN Traffic Logs

The parser can be created before or after traffic logs ingestions. In this example, we're creating the parser syslog-parser before ingestions. It starts with the creation of a file "/logParsers/syslog-parser" on the configuration file page. Then, we can follow the PAN traffic log formats documentation to implement the parser's definition.

{
  timezone: "PST"
  patterns: {
    tsPattern: "[A-za-z]+\\s+\\d{1,2} [\\d:]+" 
  }

formats: [
  {
    id: "traffic",
    format: "\\<\\d+\\>$timestamp=tsPattern$ $machine$ $future_use$,$recvdate$,$serial$,$logType$,$subType$,$confVer$,$time_generated$,$srcIp$,$dstIp$,$srcNatIp$,$dstNatIp$,$rule$,$srcUser$,$dstUser$,$app$,$virtSys$,$srcZone$,$dstZone$,$srcIface$,$dstIface$,$logAction$,$logdate$,$session$,$repCnt$,$srcPort$,$dstPort$,$srcNatPort$,$dstNatPort$,$flags$,$proto$,$action$,$bytes$,$sentBytes$,$recvBytes$,$pkts$,$startdate$,$elapsedTime$,$category$,\\d+,$seqno$,$actionflags$,$srcCountry$,$dstCountry$,$cpadding$,$sentPkts$,$recvPkts$,$session_end_reason$,$dg_hier_level_1$,$dg_hier_level_2$,$dg_hier_level_3$,$dg_hier_level_4$,$vsys_name$,$device_name$,$action_source$,$action_source$,$srcVMuuid$,$dstVMuuid$,$tunnelIDimsi$,$monitorTagIMEI$,$parentSessID$,$parentStartTime$,$tunnel$,$sctpAssociationID$,$sctpChunksSent$,$sctpChunksReceived$,$uuidForRule$,$http2Connection$,$client$,$hostchain$,$tag$,$rawMessage$,$rawSource$"
  }
 ]
} 

Alternatively, we can build the parser after ingestions. In that way, the parser syslog-parser will be available on the parsers page and we can use the actual traffic logs in the account to build the parser.  

3. Configuring PAN Firewall to Forward Syslogs to DataSet

To forward traffic logs from the PAN firewall, a Syslog server profile needs to be created from Device -> Server Profiles -> Syslog

In this example, we add two Syslog servers to this profile which both accept UDP traffic via port 514.
Next, go to Object -> Log Forwarding to create a log forwarding profile for specifying the types of firewall logs (ex. traffic) to be sent.

Finally, go to Policies -> Security to attach the log forwarding profile to your security policies in the "Actions" tab.

Commit the changes when you're finished.

A few minutes after the commit, we can go back to Scalyr and verify that the traffic logs are successfully ingested and parsed.