Grouping Messages

Sometimes a single logical message will appear as multiple lines of text in the log. For instance, Java applications often emit multi-line stack traces:

java.lang.Exception
    at com.foo.bar(bar.java:123)
    at com.foo.baz(baz.java:456)

You can group these multi-line messages into a single message. This allows sampling and redaction rules to apply to the entire multi-line unit. (Note that it's also possible to group multi-line messages using a parsing rule in the DataSet server, but this occurs after the agent has performed sampling and redaction.) To group multi-line messages, add a lineGroupers field to the log definition. For example:

{
  path:"/var/log/tomcat6/catalina.out",
  lineGroupers: [
    {
      start: "^[^\\s]",
      continueThrough: "^[\\s]+at"
    }
  ]
},

This rule creates a group whenever it sees a line that does not begin with whitespace (the start expression), and continuing through any line that begins with whitespace and the word "at" (the continueThrough expression). It will match Java stack traces in the format shown earlier.

In general, each lineGrouper has a start pattern and a continuation pattern. Whenever a log message containing the start pattern is observed, subsequent lines are then grouped together with that line according to the continuation rule.

If you have multiple lineGroupers, they are evaluated in order. The first rule whose start pattern matches a message, is applied to that message. The continuation pattern is then applied to subsequent messages from the same log.

Four different types of continuation pattern are supported. Each grouping rule should specify a start pattern, plus exactly one of the four continuation patterns:

continueThrough: all consecutive lines matching this pattern are included in the group. The first line (the line that matched the start pattern) does not need to match the continueThrough pattern. This is useful in cases such as a Java stack trace, where some indicator in the line (such as leading whitespace) indicates that it is an extension of the preceding line.

continuePast: all consecutive lines matching this pattern, plus one additional line, are included in the group. This is useful in cases where a log message ends with a continuation marker, such as a backslash, indicating that the following line is part of the same message.

haltBefore: all consecutive lines not matching this pattern are included in the group. This is useful where a log line contains a marker indicating that it begins a new message.

haltWith: all consecutive lines, up to and including the first line matching this pattern, are included in the group. This is useful where a log line ends with a termination marker, such as a semicolon.

Continue to Excluding Log Files