Relic Solution: Know when there is a notification failure

Sometimes notification fail to send, this may be due to many reasons:

  • email address bouncing/unknown/disabled
  • a downstream service is having issue and New Relic failed to connect
  • credentials no longer valid
  • invalid payload customization

New Relic now allows you to get details of these failures in the NrIntegrationError table

SELECT * FROM NrIntegrationError WHERE category = ‘NotificationError’

It is highly recommended to set an alert when any of these events occur

SELECT count(*) FROM NrIntegrationError WHERE category = ‘NotificationError’

Make sure to set the threshold to be

above 0 at least once in 5 minutes

Window Duration to

30 seconds

and Streaming Method

Event timer with 30 seconds timer duration

Finally use a different Destination than your usual notifications.