Introduction
This article will discuss the data available to you at no additional cost and will show a useful audit dashboard. Metalog data is focused on ingestion and user activity, while the Monitor log captures logs related to monitors that have been implemented, like CloudWatch. There are also Alert Logs, which are documented here.
Metalog & Monitor Logs
The easiest way to see what data is available to you within the metalog is to filter for
$serverHost = "scalyr-metalog" OR $serverHost = "monitor"
From there, if you expand the tag facet, you will see the data that is available. Out of the box, Metalog will report on logVolume, ingestion status, and user activity like logins, user page access, searches, and modified configuration files. The Monitor log content will vary based on features and integrations that have been implemented such as RDS, Cloudwatch, S3 imports, PowerQuery monitors, GCP PubSub, and other sources.
The following table provides detailed information for each tag and action.
Metalog Reference
Field: tag
|
Value |
Description |
|
logVolume |
Will return information about the log volume ingested in the account. Two metrics are collected: logVolume and logBytes. Narrow down by logBytes to see the number of bytes collected by source. This can be directly tied to billing. See the Log Volume dashboard for this data rolled up into a graph and a report for each service. It will return logBytes for each host and logfile. There is also a detailed breakdown of k8s controllers and clusters |
|
ingestionSuccess |
Will measure the max, median, mean, and p90 latency values of agent ingestion |
|
audit |
Utilizes the action field to store multiple audit level logs. See the action facet for more detailed info. And see tag.audit for a list of default options |
Action field in tag="audit" (current UI)
|
Default |
Description |
|
check file |
Logs user interactions with files |
|
getPageData |
Measures API data returned when accessing a page or a search in the DataSet UI including the user and endpoint |
|
addUsers |
Logs when a user is added to the account. |
|
saveFile |
Measures changes to a file |
|
deleteFile |
Config file deleted |
|
addKey |
API key added |
|
setAccountPassword |
Password changes |
Monitor Reference
|
Integrations |
Description |
|
cloudwatchMonitor |
When you setup CloudWatch, you can use this to check if there are errors and measure successful requests. Success: |
|
httpMonitor |
When you setup an http monitor you can determine any issues with the response much like the return body of a curl command and provide data like, statusLine, body, headers, latency, status and size. |
|
rdsLogMonitor |
Will return metadata about the RDS database and important info like errorType, error message, region, database, latency, bytesFetched, and/or linesFetched |
|
archive |
Batch export to s3 will return information about batch jobs to s3 and return info like latency, bytes, errors, and successes |
|
S3BucketMonitor |
When importing S3 bucket access logs, this will return Status Code, AWS Service, AmazonSQS, AWS Request ID, AWS Error Code, AWS Error Message |
Audit Dashboard
This dashboard tracks various user activities
{
"graphs":[
{
"query":"tag='audit' queryGroupRequest.queries contains '\"origin\":\"SEARCH\"' action='launchQuery'\n| group searches = count() by user\n\n",
"title":"Log Searches by User",
"layout":{
"h":14,
"w":20,
"x":0,
"y":12
},
"graphStyle":""
},
{
"query":"tag='audit' action='launchQuery' queryGroupRequest.queries contains 'type\":\"PLOT' not (queryGroupRequest.queries contains 'origin\":\"SEARCH')\n|group searches = count() by user ",
"title":"UI Timeseries (GraphPage) Queries",
"layout":{
"h":14,
"w":20,
"x":0,
"y":26
}
},
{
"query":"tag='audit' action='launchQuery' queryGroupRequest.queries contains 'type\":\"PQ'\n| group searches = count() by user\n",
"title":"PowerQueries per User",
"layout":{
"h":14,
"w":20,
"x":20,
"y":12
}
},
{
"query":"(tag='audit' action='launchQuery' queryGroupRequest.queries contains 'type\":\"PQ') OR //PQ\n(tag='audit' queryGroupRequest.queries contains '\"origin\":\"SEARCH\"' action='launchQuery') OR //Search \n(tag='audit' action='launchQuery' queryGroupRequest.queries contains 'type\":\"PLOT' not (queryGroupRequest.queries contains 'origin\":\"SEARCH')) //Graph\n\n\n| group searches = count() by user\n",
"title":"All Searches by User",
"layout":{
"h":12,
"w":60,
"x":0,
"y":0
}
},
{
"query":"(tag='audit' action='launchQuery' queryGroupRequest.queries contains 'type\":\"PQ') OR //PQ\n(tag='audit' queryGroupRequest.queries contains '\"origin\":\"SEARCH\"' action='launchQuery') OR //Search \n(tag='audit' action='launchQuery' queryGroupRequest.queries contains 'type\":\"PLOT' not (queryGroupRequest.queries contains 'origin\":\"SEARCH')) //Graph\n\n|parse \"\\\"filter\\\":\\\"$filter$\\\",\" from queryGroupRequest.queries \n| columns user, filter, timestamp \n|sort -timestamp \n|limit 50\n\n",
"title":"Last 50 Searches",
"layout":{
"h":14,
"w":29,
"x":0,
"y":54
}
},
{
"query":"tag='audit' action='getDashboardV2'\n| sort timestamp \n| group loads = count(), timestamp = last(timestamp) by user, dashboardName \n| sort -timestamp \n| columns user, dashboardName, loads, timestamp\n\n",
"title":"Dashboard Loads",
"layout":{
"h":14,
"w":31,
"x":29,
"y":54
}
},
{
"query":"tag='audit' action = 'addConfigFile' OR action='deleteConfigFile' \n|sort -timestamp \n\n|let action = action contains \"add\" ? \"modifyConfigFile\" : \"deleteConfigFile\"\n| columns user, action, name = name ? name : id , ip, timestamp \n\n\n\n",
"title":"Config Files Audit",
"layout":{
"h":14,
"w":60,
"x":0,
"y":40
}
},
{
"query":"action= * user = * (serverHost contains 'meta' || k8s-cluster contains 'meta')\n|group count() by user\n| columns user\n",
"title":"Logged in users list",
"layout":{
"h":14,
"w":20,
"x":40,
"y":26
}
},
{
"query":"tag='audit' action='linkAccount'\n|group links = count() by user",
"title":"Account Links by User",
"layout":{
"h":14,
"w":20,
"x":20,
"y":26
}
},
{
"breakdownFacet":"user",
"graphStyle":"line",
"lineSmoothing":"straightLines",
"plots":[
{
"filter":"count(tag='audit' action='linkAccount')",
"label":"Rate"
}
],
"title":"Linked Accounts",
"layout":{
"h":14,
"w":20,
"x":40,
"y":12
}
},
{
"query":"tag = 'audit' action='createDashboardV2'\n| group count = count() by user, dashboardName ",
"title":"Dashboards Created",
"layout":{
"h":14,
"w":60,
"x":0,
"y":68
}
},
{
"query":"tag='audit'\n| group count = count() by action\n| sort -count\n| limit 20\n\n",
"title":"Top 20 Actions (Internal and User)"
}
]
}
Comments
0 comments
Please sign in to leave a comment.