Alert Notification Service

Hi,

Below is the NRQL Query which I use to create a alert condition. So when the error goes more then 1 I should receive a mail from the notification channel. But that does not work.

Query :
SELECT average(provider.errors.Sum ) FROM ServerlessSample where provider.tag_Lambda =‘lambda_name’ where provider.errors.Sum >=1

I have also attached an image which shows my condition. Can you please suggest what changes need to be done.

Hi @lic-msd

Your screenshot does show some details, but I would need a link to the alert condition itself to be able to properly address this question.

So when the error goes more then 1 I should receive a mail from the notification channel.

When the threshold is breached, a violation should open. Whether or not a notification is sent depends on your Incident Preference setting, since notifications are only sent out when incidents open, are acknowledged, or close.

This may be the problem you’re running in to, but I won’t know for sure until I can take a look at the alert condition directly.

Hi Fidelicatessen,

I tried to change the Incident Preference setting to “By condition and entity” and also checked if any incident was open which was not there. That also didn’t work.

I have send you a private message with the link to the alert condition.

Hi,

Looks like the issue is resolved, it was the evaluation offset time which was needed to be increased as in the default 3 mins it was not able to load errors above 1. I have tried to increase it with different time interval and I was able to receive the email alert.

Thanks for the help by the way.

2 Likes

Thanks for letting us know your solution @lic-msd :smiley:

Hi,

I have Few question when I get notification alert email I get the below query report which triggered the error.

  1. :
    SELECT average(provider.errors.Sum ) FROM ServerlessSample where provider.tag_Lambda =‘lambda_name’ where provider.errors.Sum >=1 SINCE 10 minutes ago UNTIL 9 minutes ago

From my point I think 10 mins ago will be the value of the Evaluation offset if I am not wrong, but UNTILL 9mins ago why it is included in it ?

  1. :
    Also One more Question, when I get the email I don’t see the Spike on the Chart and also the time cant be seen. Why is it So ?
    Like Incident Report Time Was : 12:19
    I have attached the report when the Incident was Triggered.

Then like after some few minutes I get the report on the Graph. I have attached the graph for the same.

So how does the Time Interval Calculated. So if we have some critical error in future we should not miss it.

Hi @lic-msd

I’ll address your questions one at a time.



I think 10 mins ago will be the value of the Evaluation offset if I am not wrong, but UNTILL 9mins ago why it is included in it ?

Our Alerts evaluation system only looks at one single minute at a time. So, whatever you have evaluation offset set to will be the SINCE value, and the UNTIL value will always be one less than that – this defines the single minute which the alerts evaluation system looks at.


Also One more Question, when I get the email I don’t see the Spike on the Chart and also the time cant be seen. Why is it So ?
Like Incident Report Time Was : 12:19
I have attached the report when the Incident was Triggered.

Keep in mind that the preview chart is exactly that – it’s not a report, but rather it’s a chart to help you understand when your data stream would have opened a violation.

The reason there is a delay is due to the evaluation offset. Since the system is waiting 10 minutes before it evaluates the data (this is to ensure that all the data is present), a data spike that breaches the threshold will result in a violation X minutes later (where X is your evaluation offset value).

Since both of these questions are around evaluation offset, I would suggest reading the article I wrote about data latency (which is the entire reason you would want to set a higher value for evaluation offset). You can find that article at this link.



I hope that my answers help to clarify how evaluation offset works. Let me know if more questions crop up!

3 Likes

Hi,

Thanks for clearing my doubt. I will let you know if I have any Questions.

1 Like