Yesterday an alert came up at 1:30am regarding a failed synthetic api test.
When I checked the incident I saw that a single check had failed and that several checks every hour later were succeeding, yet the incident had not been closed.
This is the first time I experience this.
Here is the permalink. https://alerts.newrelic.com/accounts/1850063/incidents/56538625/violations?id=254847058
Validating that test yields success, so the test is working as expected. This seems to be a problem with alerts itself.
###Alerts Question Template
-
Please describe what are you seeing:
-
How does this differ from what were you expecting to see?
-
If you aren’t seeing expected alert or data, please provide a link to the incident or violation (policy, condition, data app etc.)
Helpful Resources
Relic Solution: The key to consistent alert notifications