Cannot stop browser app POSTing AJAX /jserrors/*/*/ to US endpoint bam.nr-data.net

This is related to our deleting our APM and browser apps (a topic of another support post: Cannot Delete APM)

We seem to have a high number of browser apps no longer reporting in our old US account as well as 4 that continue to report despite the fact that they shouldn’t be.

They are reporting on old APM names that no longer exist.
Not only that, the agents have been pointed at the EU endpoint and yet we are still receiving this trickle of data to the US endpoint.

The four browser apps are reporting no metrics or traffic of any kind except AJAX requests to /jserrors/*/*/ to bam.nr-data.net

We have redeployed and restarted all of our apps to not point to that endpoint and yet there are a few bytes continuing to trickle in.
We have disabled the Browser app as well as “deleted” it although neither of those have done anything noticable.

Browsing the metrics with NRQL and using SELECT * FROM PageViewTiming SINCE 1 day AGO we can see the URLs apparently being used, but when we go and check the code it’s using the EU endpoint and the newer account key in the browser script.
How is the data still being POSTed to the old account in the US?

The browser app AJAX page is: https://one.newrelic.com/-/0PLRE9GJbRa
We can’t share a public view of this page but the source shows <script type="text/javascript">window.NREUM||(NREUM={});NREUM.info = {"beacon":"bam.eu01.nr-data.net","errorBeacon":"bam.eu01.nr-data.net", ... and the Network tab in the browser shows it only POSTing and GETting https://bam.eu01.nr-data.net/events/1/

We have CloudFront in front of the sites but the application is not cached so it’s not a stale cache entry.
We’ve gone through the troubleshooting doc for removing a browser app and so far nothing has worked to prevent this data appearing.

@mikesmith Thanks for posting as I’m happy to help here! It sounds like there are a few different issues, so please let me know if I’ve missed anything.

To confirm, one issue is you have a lot of browser applications that are missing from your old account? Can you please share a link to the account and any GUID or appName/ID that you can remember from something that is missing? Are these reporting to a new account now?

Another issue is that you have 4 apps that should have been deleted or removed yet continue to report in the UI, correct? These sound like the phantom or ghost apps that were mentioned in the previous post. Can you please provide the links to those 4 browser applications?

I reviewed the one link provided for browser application “crane b2b live - crane” and checked the PageView data, and see data as recent as July 8. I wanted to check to see what I could find in the html, network console, etc., here https://quotes.brokerbility.co.uk/u/0/policies/55231 though the app is behind SSO/login. It sounds like you’ve already run a check on whether the correct script version is running on pages though.

How is the data still being POSTed to the old account in the US?

Since I cannot access the application myself, I can’t say with certainty whether some pages reporting to the browser app id above are using an old script from a different account. Another possibility is that end users have these pageUrls open in a browser window still and are sending in real data to keep these apps alive so to speak.

To clarify what you are looking to achieve, would you like to remove the 4 old browser applications from your account? And also, you’d like help in determining where missing browser applications from an old account have gone?

Please advise and we can figure out next steps here. Thank you again!

1 Like

Hi @cfrankenfield

Sorry I am not sure if I was clear.

one issue is you have a lot of browser applications that are missing from your old account.

We have the opposite issue. I didn’t mean they are “not reporting” as in we don’t see them. I meant that they are in the list and grey when we know they are not sending data. This is the case for both our US and EU accounts.

Yes you are correct about “crane b2b live - crane”. Like I say I’ve gone specifically to those pages in our system and they have the correct script code to report to the EU and yet continue to report to the US. It’s odd that they don’t actually report any real metrics though. All of the page view dashboards etc are blank or zeroed. The only real information coming in that’s visible on the dashboards is that AJAX POST to /jserrors/*/*/ to the bam.nr-data.net endpoint. That’s not even really our endpoint and so I’m not sure why it would be shown on our dashboard.

I did suspect that there might be a browser window holding that old script but I would have thought we’d see more information coming back to prove that? The URLs that we see in the NRQL query though are not really pages that anyone could or would sit on so I’m still not sure what that could be.
We’ve also tested “saved” html pages and that’s not it.

What I would like to achieve is understand why 4 browser apps were still seemingly POSTing AJAX data but only NR AJAX data.
I’ve run the NRQL today and we’re still getting pings to a couple of URLs despite those pages provably having the correct script.

I know this is probably frustrating to debug on your side as we can’t make the private pages visible to you.

I have no idea if this will help, but searching the current script for
bam.nr-data.net does show it a lot further into the imported JS in the New Relic script header.

E={beacon:"bam.nr-data.net",errorBeacon:"bam.nr-data.net",agent:"js-agent.newrelic.com/nr-spa-1210.min.js"}

This is in addition to the EU endpoints right at the start of the script. Obv it’s all been minified so it’s difficult to tell what that later section is actually doing.

This post can be closed.
We’ve deleted the old account that was the focus of this problem.
Shame we couldn’t get to the bottom of it.

@mike.smith Hi there! Apologies for the delay in response here. Is there another account or anything else we can help with?

I wanted to share that the browser agent was recently open sourced and you can see both code files and projects being worked on at the following link below:

I wanted to see if we could run a query to better identify where this traffic was being generated. I’ve used something similar in the past via NRQL with a FACET on deviceType and browserAgentVersion to see where traffic is coming from and whether it looks like cached pages.

Again, if you have applications that are in your list and they are grey as in not sending data but shouldn’t exist at all and you cannot delete them, we’d be happy to create a ticket and see if engineering is able to investigate and remove them.

Keep me posted and thank you again for your patience.

Thanks for getting back @cfrankenfield
While we have deleted our old account where this original issue was occurring we still have plenty of grey old APM and browser apps in our newer EU account that have been deleted and shouldn’t be showing.
https://one.newrelic.com/-/0nVjYv1l2Q0
https://one.newrelic.com/-/0M8jqOLllwl

This one is an example. The fleet APM is showing no metrics for 3 months and the fleet Browser app also shows no traffic. Attempting to go to the settings for the APM app just hangs with a spinning gif, and going to the settings on the Browser app just straight up errors.

This is not the only one by far. If we do create a ticket I should be able to collate a list of all the apps that I believe are no longer reporting and shouldn’t be there.

Quick side question… What’s the deal with getting direct support via tickets? We used to be able to do that but have been limited to the discussion forums for a while now. We currently have a live issue that I am struggling to get real support for. Is there a tiered support system where we have to pay for access to tickets?

Thanks @mike.smith . I’ve verified those apps are not reporting. To work this out, I created a ticket for this, to have you list all these apps, so we can delete each of them. Please check your inbox for that ticket.

Regarding direct support, the details are in Support plans.

Thanks @bkucey
Responded in that ticket and I’ve raised an enquiry with sales about the Pro account pricing.

1 Like

Glad to hear @mike.smith!