Configuring Sampling Rules

In some cases, you may not want to bring in 100% of the logs. For example, the logs could full of noisy messages and keeping a copy of every line is not critical. We often see customers who choose to retain all of their error logs but only retain a fraction of their success logs. This can be accomplished by setting up sampling rules. 

Here is a video tutorial on Sampling and Discarding data. You can skip to Sampling rules in the agent 

here

Here is an example of a log sampling rule. 

logs: [
   {
     path: "/var/log/app/*.log",
     attributes: {parser: "appLog"},
     sampling_rules: [
       { match_expression: "DEBUG", sampling_rate: 0.1 },
       { match_expression: "FINE", sampling_rate: 0 }
     ]
   }
 ]

Sampling rules use Regex which you can read more about in our DataSet Regex Reference

Continue to Server Attributes