Anomaly Detection Picking Up Nightly Jobs

Every night at 3:04am we get an anomaly detection alert for “Non-web throughput was much different than normal”. See link and image below. Our nightly jobs run at 3:00am every day. Normally anomaly detection is good at comparing the current metrics to an appropriate baseline (e.g. Sunday to previous Sundays, middle of the night to previous middle of the nights), however in this case for weeks our metrics show the same spike for ~5 minutes at 3:00am every day and it continue to alert us that it is an anomaly.

https://one.newrelic.com/nrai/feed?account=2422100&duration=259200000&state=613fd1c3-f3d3-20d3-e670-f1efcc13f043
image

Our question is: What can we do to tell/train New Relic anomaly detection that “Non-web throughput…” is NOT an anomaly at ~3am. Looking at the screenshot below we can click No for “Keep detecting anomalies like this?”, but we WANT to detect “Non-web throughput” anomalies, it’s just not an anomaly at 3am. Does clicking No only apply to 3am or will it turn of “Non-web throughput” anomaly detection for this project?

Hey there @dave63,

Welcome to the community, I hope you are well!

I am terribly sorry for such a delayed response. We appreciate all of the extra information and detail on your post, it will definitely help us get a solution for you. While your questions is a bit out of my scope we are looping in an engineer from our Alerts team to help out. They will be able to provide better context and solutions with their expertise. We appreciate your patience as we continue support.

Please let us know if you have further questions or updates from your end. I hope you have a great day!

@dave63 - When you click NO to “Keep detecting anomalies like this?” it will only apply to the system to not look at the particular time range. It will continue to detect other Non-web throughput anamolies.