Max limit on number of violations per condition

Hi folks!

Below, I’ll refer to “violations.” Keep in mind that this term is for the legacy alerting experience. The newer systems that are now available use different terminology, which I detail in this post.

Some alert conditions are capable of opening thousands or tens of thousands of violations in a single minute. This results in a situation where you, the customer, can get flooded with notifications, even causing (in some cases) your notification channels to refuse further inputs from New Relic. This also degrades system performance and reliability for you and other customers on the platform.

To prevent this from happening, and improve performance and reliability for all of our customers, we are imposing a maximum limit of opened violations per 5 minute rolling window of 20,000 violations per condition. This is not a limit on total violations, but instead is a limit on freshly opening violations, just to be clear.

If one of your conditions exceeds this limit, we will stop creating new violations for the remaining time in that rolling window. Once the number of violation initialization requests drops below 20,000 in a rolling 5-minute window we will resume opening violations for that condition.

As this is a new rate limit, we will be evaluating these settings and will adjust them accordingly. We will post updates in this thread if any changes are made.

1 Like