message field is a reserved field that is associated with each log event. It is retained as part of our lossless observability strategy. Regardless of how a log event is parsed, its original version is kept for future evaluation. Since the
message field is simply the log data we received, it does not affect your account's log volume. We've received several inquiries about modifying the contents of the
message field; for the reasons described above, we don't recommend doing so. Alternatively, there are a number of ways to customize the way log data is presented within our UI.
Please note that since these processes are performed before (in the case of redaction rules) or during (via log scrubbing rules) the ingestion event, they may impact fields that are parsed if the log structure is dramatically modified.
As mentioned in our Log Processing documentation, unwanted data can be removed from incoming log data with Data Scrubbing. This can easily be configured via the DataSet UI (User Menu -> "Log Processing")
The Scalyr Agent can also be configured to remove unwanted data from log events before transmitting them to DataSet. This is covered in our article on redaction rules.
Customize Search Display
The Search UI can be customized to display specific fields. First, click the "Display" button:
On the panel that is displayed:
- Uncheck the "Raw log" checkbox,
- Select the field(s) you wish to display in the search UI and click the "Add >" button
- Click the "OK" button to finalize your changes
The field(s) you choose can be extracted via a parser, although it / they should be consistent throughout your logs.
If the above suggestions do not work for your particular requirements, please contact email@example.com for additional assistance (as there are some additional considerations / optimizations we recommend on a case-by-case basis).