Stopped receiving Alerts in Slack

Please share your question/describe your issue below. Include any screenshots that may help us understand your question:

We stopped receiving alerts in Slack mid January. We can see that the alerts are configured in new-relic dashboard and that Slack integration is working fine as per some test that we did.

however what we noticed is that the alert template seem to have changed around the period that the alerts stopped.

I think that it is possible this might be related with some recent change to newrelic?

Hi @shiv.kandavelu, I checked recent events for the Slack channel associated with your account, and notifications seem to be sending fine. However, there has a gap from early January where new notification events simply stopped generating. Looking at the incidents on your account, I believe this is because you have an open, on-going incident that is preventing additional notifications from generating.

Alert notifications are sent out on a per incident basis, where you will receive a maximum of three notifications for every incident:

  1. Notify when the incident opens after a critical violation.
  2. Notify when a user acknowledges the incident (optional).
  3. Notify when the incident closes, after all associated critical violations have resolved.

I recommend starting with manually closing the currently open incident/violations in order for new incidents to begin flowing again. If you wish to receive more notifications in general, you will want your violations to roll up less often. You can control this by updating the incident preferences for your alert policy.

Here is a great post explaining how to make this change, and more things to keep in mind when selecting your policy’s incident preferences:

Relic Solution: Alert Incident Preferences are the Key to Consistent Alert Notifications

Let me know if you have any questions!