DNS slowing down monitors

I’m noticing across multiple monitors that the DNS lookup is consistently inflating our timings and causing inconsistent results. Here’s a graph from the last 3 hours:

These are happening across all of our monitors.

Looking at the results page for one of these shows the DNS request taking 5 seconds:

In another case it has actually caused an error and lowered our SLA. These have been happening since we signed up this week. Can we have DNS ignored in the timings if this is an expected result? We use Route53 for our DNS and have not made any changes in months.

(Since I’m a new user, I could only embed 2 images and I couldn’t link to any others.)

Hi @jameshartig Thanks for reaching out and providing the screenshot. After further investigation on these timings reported similarly from other customers, we have been working with our upstream provider to resolve this issue. Since then, we have observed improvement in DNS times from the Dallas, Fremont, and Newark locations resulting from a fix implemented on March 29th.

Our Atlanta location should now have similarly lower DNS resolution times, beginning around 9AM PT yesterday (4/3). We’re continuing to work with our upstream provider and monitor DNS request timings at these locations, so let us know if you’re continuing to see these high DNS request times.

It looks like the screenshot above was taken on the 30th – is this issue still occurring from any location(s) in particular, or all locations?

There is no present way to ignore (not capture) DNS timings, though you can click the DNS lookup button below the chart (pictured above) to show adjusted timings without this timing. Additionally, all request timings are broken out into individual attributes of the SyntheticRequest Insights event, allowing you to generate your own reports within Insights that could exclude these timings. Additional details on the Synthetics data available within Insights can be found here:

https://docs.newrelic.com/docs/insights/explore-data/attributes/synthetics-default-attributes-insights

We had 5 second DNS lookups happen twice today both from Atlanta, GA, USA.

I appreciate the transparency on this issue.

1 Like

Hi @jameshartig thanks for the update! Despite the overall improvement from the 3rd, we are also continuing to see instances of these 5 second DNS lookup times on our end. We’re continuing to investigate, and I’ll post a reply here once we have an update on the issue.

@btribbia We are seeing a similar issue now.

Hi @jgbutterworth Love the handle and profile pic! Could you please send over a permalink to where you are seeing this issue so that we can take a deeper look?

https://one.newrelic.com/launcher/nr1-core.explorer?platform[accountId]=949064&platform[timeRange][duration]=86400000&pane=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb25zIiwidHJhbnNhY3Rpb25JZCI6IjViMjI1NzY1NjI1NDcyNjE2ZTczNjE2Mzc0Njk2ZjZlMmY0NTc4NzA3MjY1NzM3MzZhNzMyZjQ3NDU1NDJmMmY3NzZmNzI2YjY2NmM2Zjc3MmY2MzYxNzM2NTVmNjQ2NTc0NjE2OTZjMmYzYTYzNjE3MzY1NWY2OTY0MjIyYzIyMjI1ZCIsInRyYWNlSWQiOiJkODY5ZGM0Zi03NDgxLTExZWMtODZiNi0wMjQyYWMxMTAwMGVfMF8yNTY3IiwiZW50aXR5R3VpZCI6Ik9UUTVNRFkwZkVGUVRYeEJVRkJNU1VOQlZFbFBUbnd6TVRVd09UYzFNUSJ9&sidebars[0]=eyJuZXJkbGV0SWQiOiJucjEtY29yZS5hY3Rpb25zIiwiZW50aXR5R3VpZCI6Ik9UUTVNRFkwZkVGUVRYeEJVRkJNU1VOQlZFbFBUbnd6TVRVd09UYzFNUSIsInNlbGVjdGVkTmVyZGxldCI6eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb25zIn19&state=d6e2a267-113f-b44d-3fc7-030f9a00e593

Hi @jgbutterworth It looks like you linked to an APM app. I’d suggest bringing this topic to that category: Topics tagged apm

1 Like