Alert on drop of events compared with previous 24 hours

Hi,

We are trying to set up an alert that compares the total events from the previous 24 hours to the the total number of events in the 24 hours before that. I can run a query that returns the percentage difference using SINCE and COMPARE WITH but it seems we cannot use these keywords when creating an alert condition.

This is the query I am using in Insights:

SELECT COUNT(*) FROM main_decision WHERE appName='main-service' SINCE 1 day ago COMPARE WITH 1 day ago

We want an alert when the events drop by 30% or more. Is this possible and could someone point me in the right direction?

Thanks,
Bern

Hey @bernard.oneill - COMPARE WITH is not currently supported with NRQL Alerting.

An option that may work is Baseline Alerts, which allow you to set a threshold that will alert if your metric goes too far beyond the standard range for the given time period.

Baselines learn from your data to build a good baseline value to compare against, any major increase or decrease in that value will deviate from the baseline enough to trigger a violation.

Is that something you could explore?

Hi Ryan,

Thanks for the reply. Unfortunately baseline alerts will not work for us either as we use terraform to configure new relic. Is there another approach we could use to export the result of the NRQL query as a metric? If we could export the metric we could easily configure the alert.

Thanks,
Bernard

Hmm - That could be doable.

You could write a Synthetics Script that uses the Insights Query API to pull out the values Current and Previous.

Then you could work on - in the script - finding the difference of the values, which you can then send to insights again using:

var diff = previous - current;
$util.insights.set('diff', diff)

Then create a NRQL alert on the query:

SELECT average(custom.diff) FROM SyntheticCheck 

There’s an example Synthetics API Script here for that here: https://docs.newrelic.com/docs/synthetics/new-relic-synthetics/scripting-monitors/write-synthetics-api-tests#get-insights-example