Preparation
Parsing
Before configuring alerts, confirm that the data you need from your log events is readily available. This is as simple as:
- Modifying parser(s) to extract values from log events of interest
- Assigning attributes / id values as necessary to quickly isolate log data via the alert
- Time zones are of particular significance. An incorrect time zone could result in false alarms, or alerts failing to trigger.
- First, Identify the time zone associated with the timestamps contained in your log data.
- Then, set the time zone associated with the log data manually within the parser, or extracting it from the log data (see: https://app.dataset.com/help/parsing-logs#specialAttrs).
- Verifying that your parser is correctly interpreting the timestamp associated with the logs
A combination of the above factors are typically the culprit when alerts don’t behave as expected.
Alert Syntax
When specifying field names (for example, $timestamp
, $parser
, $yourField
) in alerts, we recommend including the $ sign in your description clause.
Q: I want to trigger an alert if $queue_size
reaches 5000 more than 10 times over a 24 hour period
A: The alert trigger would be something like:
count:24h($serverHost==”HOST” && $metric==”queue_size” && $value > 5000) > 10
Where $metric
and $value
are extracted from the log events by a parser
Q: Alerts are no longer sent to my email address, yet they previously were.
A: It’s possible that the email account rejected or bounced an incoming alert email. Once this happens, our email platform will refrain from sending further notifications. Please contact support@dataset.com to investigate, and whitelist alerts@dataset.com on your mail server from any possible spam recognition or rate limiting in the future.
Slack
Q: Are the fields provided with Slack alerts modifiable?
A: To an extent, yes. There are several built-in fields that can be used to modify the structure of Slack alerts. See the “Substitution Tokens” section of https://app.dataset.com/help/alerts. Also, you’ll want to set up your Slack alerts via webhook. The resultant alertAddress
field is:
webhook-trigger:POST https://hooks.slack.com/services/XXX/YYY/ZZZ[[{\"text\": \"<@userId> #title# <#link#>\",\"username\": \"scalyr\"}]]
Grouping
Q: Is it possible to dynamically configure alerts for a particular list of servers?
A: Yes! This can be done for a list of servers (or all servers). See: https://app.dataset.com/help/alerts#alertTemplates
PowerQueries Support
Q: Can PowerQueries be used to construct alerts?
A: Yes -- this is possible via a PowerQuery monitor. See: Building PowerQuery Monitor
Comments
0 comments
Please sign in to leave a comment.