How to Breakdown Log Volume by a Custom Field

DataSet has a default log volume dashboard that shows log volume breakdown against hosts and K8s cluster (if applicable).

https://app.scalyr.com/dash?page=log+Volume

It provides convenience to the end-users to monitor their log volume. However, as the log volumes increase on the account, you may need more insights on what the major contributors are and require a breakdown against terms other than the host.

This can be easily accomplished by leveraging the join statement in PowerQuery. All you need to do is to join the two tables by serverHost/k8s-cluster, and group by the custom field again to get the complete picture of the log volume breakdown by any custom field in your logs.

For example, I want to break down log volume using the field serverIp in my logs. I can simply use the following query to get the answer.

|join
vol = ($tag == "logVolume" metric='logBytes'|group MB = sum(value)/1024/1024 by host = host? host : k8s-deployment|sort -MB),
ServerIp = (serverIP == *|group serverIP = any(serverIP ) by host = serverHost)
on host
| group mb = sum(MB) by serverIP 
| sort -mb

Note: keep in mind that the selected custom field serverIP needs to have a 1:1 relationship with the host to get an accurate result. Otherwise, selecting any(serverIp) will grab a random IP from the set and attach it to the host. You can use the query serverIp = * | group estimate_distinct(serverIp) by host = serverHost to verify the 1 to 1 mapping relationship.