The log volume ingested by DataSet is the basis for billing, so it's important for customers/prospects to understand how this volume is calculated, in order to optimize your logs and agent configurations to get the best value out of the service.
Here is an explanation of what is counted toward your log volume and what is not:
- Raw log messages are counted
- Metalog events are not counted
serverInfofields (app, launchTime, parser, session, machine, serverType, serverScope, sessionType, serverHost) are not counted
- The logfile attribute, as well as internal k8s attributes ("containerName", "containerId", "pod_name", "pod_namespace", "namespace", "pod_uid", "k8s_container_name", "original_file", "scalyr-category", "k8s_node", "container_id") aren't counted
- Other than the above, attribute values that originate in the Agent or at the DataSet API are counted. Attribute name lengths aren't counted, but each attribute costs 1 byte. (Attributes that are created by the parser are not counted.)
- There is no separate charge for the contents of the message attribute. You are only billed once for the original log event.
- Having no attributes would theoretically cost 1 byte