Huge lag time in Deployment Notifications

We have Deployment notifications set up to post to Slack when a deployment happens

This has been working without issue for over a year, but as of last night we saw almost 12 HOURS lapse between when a deployment occurred (~4:30PM EDT 2/4/2021)…

…and when we ultimately got the Slack notification (~3:45AM EDT 2/5/2021).

Is there a known issue with this service causing this delay?

1 Like

Note when clicking the “send a test message” link in the Deployment Notifications page above we get the message in Slack immediately.

Same here. Actually even worse… We did see a huge delay first, but since last week, no recorded deplyoment makes its way to Slack. Testing the web hook succeeds, deplyoments aren’t notified on Slack

We upgraded our deployment registration call to the NewRelic API v2 but to no avail.

Is this a known issue with you guys?

Same here @philipp.gysin. While initially there was a delay on when we got the deployment notification, now these have simply stopped working altogether.

Hello all,

Thank you for reporting this issue with webhook notifications related to deployments. This does appear to be an issue with our product; our developers have committed to prioritizing this as soon as they are able to reasonably do so.

We appreciate your input on this issue and we hope to have a solution in place soon. I can’t say for sure how long this will take to be implemented, but I can assure you that we take our commitments seriously. If you have any further questions or comments regarding this, please let us know.

3 Likes

@ntierney this seems to be happening again. We see our deployments recorded correctly in the New Relic UI, and clicking the “send a test message” link on the Deployment Notifications page we get the message instantaneously in Slack, but we haven’t received a real notification in Slack since Sept 2 (and we deploy multiple times per day)

1 Like

Same here. We don’t receive deploy notifications via the Slack webhook anymore. Last notification received was on 22nd August 2021. Sending a test message works fine.

Hey folks - sorry for taking so long to reply, but it does appear that we were able to address the issue that had been causing this behavior. Please let us know if it’s still an issue for you so that we can keep working with you.