Introduction
Update: The settings provided below are the default values for Scalyr Agent releases starting with v2.1.6. Please see the associated release notes for more details. No manual configuration is required in order to achieve the higher throughput from these settings. If you wish to preserve the legacy settings, you may set the max_send_rate_enforcement
option to legacy
. More information is available in the v2.1.6 release notes. The contents of this article were retained for historical / legacy purposes.
Optimizing Scalyr Agent settings to reduce skipped bytes
The Scalyr Agent's default settings are somewhat conservative to prevent the consumption of excessive server resources. This also helps avoid situations where a configuration bug leads to massive amounts of logs being generated, unexpectedly driving up your DataSet costs. The trade-off is that logs which are prone to significant bursts of data can overwhelm the Scalyr Agent and cause it to skip logs. If you are seeing skipped logs messages in your agent.log, you may wish to consider implementing these settings.
The easiest way to check if the agent is skipping logs is to search for "skipForTooFarBehind"
logfile="/var/log/scalyr-agent-2/agent.log"
. If there are no results, then no bytes have been skipped. Otherwise, each log line in the results will show which logs had skipped bytes, and how many bytes were skipped in each log.
The following settings should allow a single agent to achieve around 5MB/s of raw log throughput.
If you are still seeing skipped bytes after implementing these settings and wish to capture that data in DataSet, please contact support@scalyr.com for further assistance.
Examples
The following pages provide settings in two different formats: agent.json and Kubernetes config map.
For more information on agent configuration see the following:
- Configuring the Scalyr linux agent via agent.json
- Configuring the Scalyr Kubernetes agent with ConfigMap
- Environment-aware variables in a ConfigMap
- Default scalyr-agent-2 ConfigMap
agent.json settings (paste beneath API key declaration in agent.json configuration file):
YAML ConfigMap settings
Comments
2 comments
Thank you for updating this Article! To help others not have to search for the release notes:
@Lutz
Thank you for the helpful suggestion. I have added a link to the release notes as you suggested.
Please sign in to leave a comment.