Alerting in missing daily traffic

We have a daily job that generates traffic that we can see in Metric events.

I am trying to create a condition that will trigger an incident if job has not run.

I am really struggling as to how to set this up as clearly it cannot be a simple condition against a threshold. I have tried working with muting rules but so far to no success and also I tried generating some values for periods when the traffic is not occurring in NRQL e.g. the below but that doesnt return any results (as it is adding a number of null I guess)

SELECT filter(max(newrelic.timeslice.value) , where hourOf(timestamp) rlike r'^[12][0-9]:00$' or hourOf(timestamp) rlike r'^[016789]:00$' ) +filter(max(newrelic.timeslice.value)-min(newrelic.timeslice.value) , where hourOf(timestamp) rlike r'^[2-5]:00$' ) FROM Metric WHERE metricTimesliceName='JMX/queue/irisShiftInformationQueue/EnqueueCount' and entity.guid = 'MTQ1NzMxOHxBUE18QVBQTElDQVRJT058NDgxODU3MDgx' since this week limit max timeseries auto

Any suggestions welcomed

@stuart.elvins - It sounds like your alert condition may not be opening a violation due to NULL values. Since you are trying to open a violation when a job has not run, I am assuming you are using something like less than 1 which would require a Loss of Signal be set up.

Take a look at this article, which will explain why count() and uniqueCount() will never return a value of 0. The article also goes into a couple possible solutions depending on your use-case – these are new features which were also released with the Streaming Alerts Platform.

What I would recommend with this condition (and possibly others like it) is to set up a Loss of Signal (LoS) for the same duration as the threshold, and configure it to Open new "lost signal" violation when met. This will result in violations opening when the query stops returning numeric values and drops to 0/NULL.

I hope this helps!

Thanks @dkoyano

So I have evolved the approach

The current query is shown below
SELECT
filter(max(newrelic.timeslice.value)-min(newrelic.timeslice.value) , where hourOf(timestamp) rlike r’^[1-3]:00$’ )
FROM Metric
WHERE metricTimesliceName=‘JMX/queue/irisShiftInformationQueue/EnqueueCount’
and entity.guid = ‘MTQ1NzMxOHxBUE18QVBQTElDQVRJT058NDgxODU3MDgx’ since 12 hours ago timeseries

In query builder I see

As you can see no signal is generated before 02:00am nor after 05:00am which is in line with the filter clause

But when I put that NRQL into a policy condition it looks like 0’s are being returned even when there is nothing being generated from the NRQL

Do you know why?

1 Like

According to the task in my work, I’ve also figured out this problem. The main reason why we don’t see values in the preview plot (those where we set an alert) is that this preview is using by default parameters: Since 6 Hours Timeseries 1 minute.
It’s not possible simply to set an alert if your data is uploaded to New Relic once a day, due to limitations in agg window and these default parameters from the alerts preview part.

@stuart.elvins please tell me, have you already fixed your problem?
And also guys from NewRelic please answer here

1 Like