AWS Metrics Reporting Invalid Values

Support: Alerts/Applied Intelligence (AI) Alerts
, ,
#1

We have been getting spammed today with invalid alerts coming from New Relic. They are all being triggered by AWS metrics (primary load balancer healthy hosts and database burst balance). This has been triggering our PagerDuty integration and causing our team a lot of headaches.

The data that NewRelic is alerting on is not correct. When viewing the same metrics in AWS, AWS shows no issues. I can confirm that AWS is correct in this case because if NewRelic was right, our service would legitimately be down all day.

Any help with this issue would be greatly appreciated.

#2

Hello @kevin32

Welcome to the community and thank you for reaching out to us.

I understand that you are seeing some “false” alerts for AWS services in New Relic alerts for AWS services like ELB, Hosts, and Database burst.

Can you please provide the following so, that I can have a further look at it?

  • Screenshots from the AWS dashboard showing the metrics on the time frame that those alerts went off, so I can compare it with the New Relic alerts.
  • Permalink. for those alerts you had mentioned.
  • Account provider that you are referring to

With the above, I can double-check and see if I can narrow down any indication of the issue on our side or possibly a cause for these false alerts.

Please do not hesitate to contact me in case of any additional queries or issues. I will be happy to help you.

Kind regards,

#3

Hi @vhenrique ,

Thanks for helping me look into this issue.

Burst balance alert:

Permalink: https://onenr.io/0BQ1A0YPlQx
AWS screenshot in PST:

Screen Shot 2022-05-18 at 3.20.29 PM
Screen Shot 2022-05-18 at 3.20.29 PM2478×1060 226 KB

Healthy hosts alert:

Permalink: https://onenr.io/0oQDWb7VDjy
AWS screenshot in PST:

Screen Shot 2022-05-18 at 3.23.37 PM
Screen Shot 2022-05-18 at 3.23.37 PM2466×1068 201 KB

These alarms went off multiple times yesterday (5 or so times for each of those metrics).

#4

Hey there @kevin32,

We appreciate you providing the extra information, links, and screenshots. I apologize for the delayed response as we are still working on this for you. We will reply here with an update soon. We appreciate your patience as we continue to provide support.

#5

Thank you @michaelfrederick . Looking forward to your update on this issue.

#6

Hey @kevin32,

You are very welcome. We will provide an update shortly. Please let us know if we can assist with anything else as well.