Q: We need to trigger an alert when a field
a increases its value over 5 times within 10 minutes. Can I set up an alert for it?
A: Yes, you can try the following alert's definition to achieve a similar outcome.
max:10m(a where serverHost == 'myHost')/ min:10m(a where serverHost == 'myHost') > 5
You'll need to substitute the example filter
serverHost == 'myHost' with one that is specific to your platform (along with any other applicable conditions).
This expression computes the min value and max values of field
a and divides the two numbers. We can then use the result to verify whether
$a has increased by more than 5 times in the last 10 minutes.
Q. Can I create an alert that checks a field's value using every 5-minute as a unit? If the field's average value in the last 5 minutes is more than the 5 minutes prior, it sends an alert notification.
A: Let's say the field's name is called
customEvent=="QueueTracker" is an example query statement that identifies the specific log lines.
Then, use the values in a derivative statement:
sumPerSecond:5m(qSize where customEvent=="QueueTracker") / (sumPerSecond:10m(qSize where customEvent=="QueueTracker") - sumPerSecond:5m(qSize where customEvent=="QueueTracker")) > 1
The above alert definition triggers on the smoothed sum of
qSize over the past 5m is 1x (or more) greater than the smoothed sum of
qSize from 5 minutes prior. By subtracting the numerator (5m) from the denominator (10m), we effectively constrain this equation to the 5 minutes prior to the current 5-minute interval.