Alert based on custom events not triggering violations

Support: Alerts/Applied Intelligence (AI) Alerts
,
#1

Hello, the linked alert condition never triggers a violation, not even with signal loss enabled:
https://onenr.io/0M8jql2J0Rl

It’s a simple SELECT count(*) FROM CustomEvent WHERE field = 'some-string' .
I’ve tried different aggregation windows, evaluation offsets, etc, however it still never triggers.

Any idea what I might be doing wrong?

Also, docs are a bit unclear on how signal loss and gap filling work together - if I define a gap-filling value, does that mean that “signal loss” will never trigger?

Thanks!

#2

Hi @ivan.zamanov - The alert condition is based on a NRQL alert condition using count(). New Relic used to insert synthetic zeroes as a result for some queries, but ever since Streaming Alerts for NRQL conditions was introduced, synthetic zeroes are no longer inserted (the rules for when they were inserted was obtuse and opaque, so now you get exactly what the query returns).

Take a look at this article, which will explain why count() and uniqueCount() will never return a value of 0. The article also goes into a couple possible solutions depending on your use-case – these are new features which were also released with the Streaming Alerts Platform.

What I would recommend with this condition (and possibly others like it) is to set up a Loss of Signal (LoS) for the same duration as the threshold, and configure it to Open new "lost signal" violation when met. This will result in violations opening when the query stops returning numeric values and drops to 0/NULL.

I hope this helps.

1 Like