Data Scrubbing

Introduction

The web-based Data Scrubbing routine that is referenced by this document is accessed via the User Menu -> "Log Processing" of the DataSet site.

In function, it is similar to configuring redaction rules for the Scalyr Agent (for more on that topic, see Redaction Rules and the video at Configuring Redaction). While the Scalyr Agent's redaction rules apply to logs uploaded by a single instance of the Agent, Data Scrubbing can be applied to multiple log sources.

In either configuration, scrubbed data is not counted against the account's log volume since:

1. The Scalyr Agent performs redaction before logs are sent to DataSet
2. Data scrubbing via the DataSet site occurs before the log volume is tallied

Configuration

Click the User Menu -> "Log Processing" option

Enter the filter name and a DataSet search query to filter logs. Be sure to test the filter!

For the "Match Text", please refer to this guide. Please note that special characters (ex. \d, \w, \s, etc.) are single escaped (by one backslash \).