This article pertains to the K8s Agent. As mentioned in our K8s Agent documentation, the Agent captures stdout / stderr output from pods within the same node and uploads them to DataSet. For ease of use within the K8s environment, a number of common Agent functions can be configured via annotations. This article explains how to assign a parser to pod logs and include / exclude logs by using annotations. More information can also be found here.
Note: Substitute pod-name-xyz123 with the full name of your application pod
Set the parser via the CLI
The "docker" parser is the default parser for logs which originate from pods where a parser is not explicitly set. The "docker" parser works well if your logs are in JSON format and no modifications need to be made. We generally recommend setting custom parsers if:
- Your logs are an unstructured format (ex. CSV)
- Specific values will be extracted from log events
- Key-value pair extraction will be used
- Logs consist of different formats. For example, you have HTTP access logs, application logs, and firewall logs. In this case, 3 parsers would be created, one to accommodate each log format.
Using custom parsers improves the performance, maintenance, and scalability of your DataSet configuration. Don't use a single parser to process multiple log formats!
Set the parser to "logGenTest" with an annotation
kubectl annotate pod pod-name-xyz123 --overwrite log.config.scalyr.com/attributes.parser=logGenTest
Set the parser via the deployment metadata
The parser attribute can also be set via the application pod's deployment configuration
kubectl get deployments --show-labels
Add the annotation to the
template section of the deployment metadata, then apply the updates:
Confirm that the parser attribute was correctly set
kubectl get pods pod-name-xyz123 -o custom-columns=ANNOTATIONS:.metadata.annotations
This should return output similar to
map[... log.config.scalyr.com/attributes.parser:logGenTest ...]
Edit or create the "logGenTest" parser within the DataSet UI by clicking the User Menu -> "Manage Logs" -> "Parsers"
Any changes made to the parser will be applied the associated log events as they are ingested
Unset the parser attribute
kubectl annotate pod pod-name-xyz123 log.config.scalyr.com/attributes.parser-
This command will unset the parser attribute. The default "docker" parser will be applied to the associated log events.
Only upload logs from specified pods
Logs can be restricted to specified pods in order to reduce chatter and log volume. Similar to configuring the parser, this is achieved by setting annotations.
SCALYR_REPORT_CONTAINER_METRICS: "false" to the K8s Agent ConfigMap
Restart the K8s Agent for the change to take effect
kubectl delete pods -n scalyr --field-selector=status.phase=Running
Set the following annotation to include logs from specific pods
kubectl annotate pod pod-name-xyz123 --overwrite log.config.scalyr.com/include=true
or add via the deployment metadata
This setting will only upload logs from pods where the
include attribute is set.