How are New Relic browser types detected and can I graph this?

Hello - fairly new user to NewRelic, which has been installed on our web application for a while but was never properly used.

We are trying to get better insight into usage patterns, perf problems and also create some useful dashboards for application operation.

We have an APM Pro account, and I’m not totally sure how it was setup - I believe via APM. Our application is PHP based and I’m trying to understand how to figure out the breakdown of browsers using our app.

Can NewRelic help me with this (and avoid me having to incorporate google analytics?)

I though I might be able to do some NRQL like: SELECT count(userAgentVersion) FROM PageView FACET userAgentName SINCE 3 DAYS AGO TIMESERIES

But this gives me incorrect results, incorrectly reporting IE as the majority of the browsers, but this isn’t correct and I think it has to do with how user agent strings are reported by browsers.

I thought I had seen a stat on browser breakdown somewhere but I can’t find it - and am wondering if I made it up.

Is there a way to use NewRelic to help with this?

On a similar note - can NR help me understand how many users are using the app at the time I am getting stats like browser response time or backend processing timings (without adding a customer metric). I thought the EndUer/Session metric might give that, but it seems like a lower figure than I was expecting - so I’m questioning whether it is what I am after.

If you could point me to the right areas it would be much appreciated. I find the product powerful but more complicated than I would, in particular with all these different legacy, new, one, insight technologies - how am I supposed to understand what I’m supposed to use to look at data and create dashboards?

Tim

Hi @tim.mackinnon1

Let’s see what NRQL we can come up with!

How many sessions are active while I’m looking for browser response times and backend durations

SELECT uniqueCount(session), average(duration), average(backendDuration) FROM PageView SINCE TODAY FACET hourOf(timestamp) 


Count of sessions per browser

SELECT uniqueCount(session) FROM PageView SINCE TODAY FACET userAgentName, userAgentVersion, userAgentOS 

I understand this is the problematic one for you - you are not expecting IE to be as popular as it is with your site - so I want to pause here and chat about this data.

The browser snippet pulls out userAgent details and ships that to NR, where on the backend we are translating that to a browserName, browserVersion, and OS.

We have seen very rare issues in the past with this backend translation, where Microsoft’s Chromium based Edge browser had sent it’s user agent as Edg, but we expected Edge, thus for a short time Edge was tracked as Chrome.

But these kind of issues are very rare, and I can’t imagine what could be misinterpreted as IE.

In fact I just tried to open up a test site in Safari, Chrome, and Firefox on MacOS, and also Chrome and Firefox on a Linux VM. All of these PageViews came in with the right UserAgents.

So - with all of that said, do you have information to suggest that IE would not be popular among your site’s users? If we can prove that the NR reporting is wrong, we’ll be better able to get a bug reported. :slight_smile:

Hi Ryan, thanks for such a comprehensive reply- I learned a lot from it, and am only just getting to grips with NRQL and what is possible (is there something that better describes the difference queriable attributes, or does it just come from seeing examples and learning whats possible?).

However, back to the bottom bit, and the counts we are getting - I am pretty sure the data coming back in NR is incorrect.

From your query, running it on our data today: I see IE 11 having 56% (Windows) of all counted sessions, with Safari 33% (but OS Linux?) and Chrome 6% (Windows). The IE figure seems very high to us - but it turns out we have some legacy system stats where we log the userAgent string in a db-table, and interestingly it doesn’t seem to match up with the query you gave me. This makes me wonder if NR is parsing something incorrectly (or we have something out of date, or setup wrongly?).

I’ve gone and queried the data from that database - and picked out the IE sessions which I believe are like this (e,g. it says Trident, and not Chrome)
“Mozilla/5.0 Windows NT 10.0 WOW64 Trident/7.0 rv:11.0 like Gecko”

Vs. the Chrome ones like this:
“Mozilla/5.0 Windows NT 6.3 Win64 x64 AppleWebKit/537.36 KHTML, like Gecko Chrome/84.0.4147.89 Safari/537.36”

And with my dataset - it shows that IE should report 3.4% not 56% and chrome is like 90%

Given many of the NewRelic metrics show browser type related breakdowns (e.g. throughput by browser) - it seems weird that this would go undetected, so maybe we are doing something wrong?

I do note that with NR being sample based - it appears that the counts that NR comes back with are about 50% of the true data - but I think that would be normal right?

Is there something else I could double check to verify this? I want to make sure that we are getting reasonably accurate data back from the product, otherwise we could start making very bad decisions.

Tim

Hi, @tim.mackinnon1: The default events and attributes are all described in the documentation. Start here: https://docs.newrelic.com/docs/insights/event-data-sources/default-data/events-reported-new-relic-products, then select a product, such as Browser: https://docs.newrelic.com/docs/insights/insights-data-sources/default-data/browser-default-events-insights.

That page lists the default events sent by New Relic Browser; when you select an event type, such as PageView, you can see the default PageView attribute names and descriptions.

1 Like

Hey @tim.mackinnon1

quick note on:

is there something that better describes the difference queriable attributes, or does it just come from seeing examples and learning whats possible?

A couple of resources I’d recommend:

  • New Relic University has a lot of NRQL resources, here’s a link to some search results
  • Attribute Dictionary, poke around to find the queryable attributes from each event type.
  • NRQL Reference outlines all of the different clauses of a query
  • I don’t have a link to this, but, I suggest viewing data in New Relic One. Most if not all New Relic One charts are built with NRQL, so each has a 3 dot ... button that you can click to hit View query, that’ll show you how we build the charts.

UPDATE: Looks like @philweber got there before me with the NRQL resources :smiley:


As for the User Agent issue, it may well be that your UserAgent of:

 “Mozilla/5.0 Windows NT 10.0 WOW64 Trident/7.0 rv:11.0 like Gecko”

is being mistranslated on our side. I’m going to tag in @hwilkalis here, Holly is one of our browser experts and may know from memory if this is a recognisable issue. If not, I’m sure Holly can help more than I can.

Thanks for the quick replies - hopefully @hwilkalis can shed some light. i can certainly give a list of unique user agent strings that we have seen today, which may help diagnose this or help me understand what we may have configured wrong (as I mentioned, we inherited the setup of this, and it had not been used much - which is a shame given the potential to diagnose and make much more informed decisions, so I’m keen to rectify this and learn/teach our team how to make better use of this stuff).

Tim

Hey @tim.mackinnon1,

I always hesitate to call myself a Browser expert :slightly_smiling_face: but I think I can answer your questions, or at least give you some things to think about.

First, Phil and Ryan are correct that we do some behind-the-scenes parsing of the user agent string data that we collect before we report it to you. We do this to try and provide some actionable groupings - with so many possible variations in the user agent strings, if we gave you only the raw string you really wouldn’t be able to group in any any meaningful manner.

It is possible that we’re incorrectly attributing one or more user agent strings. I have ways of poking at the data in our consumer to see whether that might be happening and I’ll take a look later today.

However, in the meantime, I noticed that a lot of the queries that you’re referencing are based on the session identifier, such as uniqueCount(session). Does this same mismatch of IE vs. other browser hold up if you’re counting only PageView events, for example SELECT count(*) from PageView FACET appName ? I ask because that session ID value is set by the Browser cookie, and many newer browsers and extensions could be blocking third party cookies and keeping us from being able to assign a session ID value.

1 Like

I wound up having 5 minutes here at the end of my day so I was able to do a check to validate that example user agent string you provided, Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko, to see how it’s being parsed by our consumer.

It appears you are correct, traffic with this string is being reported as Windows IE 11. I checked it against some other user agent lookup tools as well, like https://developers.whatismybrowser.com/ and https://userstack.com/. They’re also translating that string to IE 11. So what we are reporting is at least consistent with other services.

If you think this or other user agents should be identified differently we can certainly submit a feature request for you with how you propose that they be handled. Worth mentioning though that you can also grab the raw user agent string and report it, unparsed, as a custom attribute using setCustomAttribute().

Hi @hwilkalis thanks for looking at this for me - however I wonder if I have confused you on this.

My account - is showing that all our traffic is predominately IE browsers - even the built in NewRelic reporting (e.g. top browsers by throughput).

When I run a query like: SELECT count(userAgentName) FROM PageView FACET userAgentName SINCE last month

It shows that IE is 77% of my traffic and this is WRONG - my own data (as it turns out we are logging this info in a traditional way) indicates that IE should be more like 3%, and Chrome is 90%.

Now it could be that my account is broken in some way? But I am deeply suspicious of how this is being presented by NR at the moment. I have also now noted that the same applies for session data, so it could be something more deeply routed in our setup now I’ve dug into it more.

When I initiated this, I didn’t have the session example - and so I wondered if something was up with the parsing that would cause this - but now is it possible that whatever NR injects into our pages is broken?

It sound like the queries I have been using are supposed to work - but do they work in a PHP system? (it is possible you are all using something else and getting valid results, where I am not)?

Tim

@hwilkalis I realised that I didn’t try out your suggestion - I thought it might reveal the issue at play - as the count is very low I get only 267 counts for my primary app (and 9 for an admin app) - but then when I adjust the query to

SELECT count(*) from PageView FACET appName since 1 week ago

it now shows my primary app has 10k results… which could be correct - what is the default if you don’t specify a “since” clause - is it 1 hour?

I am very confused and now mistrustful of the data I am getting from NR.

How is the best way to diagnose all of this - I feel like I am getting out of my depth a bit, and something that should just work - isn’t.

Tim

Answering my own question - I think that not specifying a SINCE clause defaults to 1 hour.

Then my earlier anaylsis on a days worth of data stands correct - my manual parsing of our log files shows 90% Chrome, whereas NR suggests 77% IE (almost the exact opposite).

So how can we explain this?

Tim

Correct, @tim.mackinnon1, if you don’t include a SINCE clause it defaults to one hour.

Did you see my second update where I confirmed that the user string you reported is being translated to IE 11?

I saw that - but wasn’t clear on what you meant, as my point was that the genuine Chrome entries in our manual logs (the non-Tirdent ones) are not being reported properly by NR, its got the % totally incorrect?

However, I have been in contact with NR support and it appears that our browser scripts might be configured to use the wrong product (we have Lite and not Pro)… I’m still not sure how/if this would impact a query based on a days worth of data and totally get the proportion of browser distribution wrong… but I’m willing to entertain this line of thought to rule out one variable, but it sounds a bit fishy to me. Still, as i am very much a novice in all of this - and just want to get some reliable data and show that NR is actually helping us, then I’m a bit at the mercy of whatever I’m told.

I do appreciate the help everyone has given me to date - I’d have written off the product by now otherwise.

Tim

Hello @tim.mackinnon1,

I haven’t looked at the ticket you created, though I can say that regardless of the agent type and subscription, the user agent string data is going to be processed the same way. So while subscription and retention might affect how much data you have, it won’t affect the accuracy of it

I may be a little confused myself by now. I was looking at the user agent string that Ryan called out as being potentially mistranslated, which was Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko, and I confirmed that it is being interpreted as IE.

I looked for the other string you mentioned just now as well, Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36 Edg/84.0.522.44. I’m not finding anything identical to it, there are some that are similar but very few.

I do note that with NR being sample based - it appears that the counts that NR comes back with are about 50% of the true data - but I think that would be normal right?

Browser actually does not sample data, what we report out to you is what we collect, and assuming that the agent is working correctly (for example, instrumentation is working, agent isn’t being blocked by ad blockers, etc.) what we’re reporting should be a complete representation of your users.

In case it helps, I did pull out a recent summary of our consumer data showing the raw user agent data we received on your account matching any variation on Gecko and what browser we mapped it to. This data is straight from our consumer (which we do sample), so the counts the counts may not be accurate but they should be at least proportional for the time period in question. The JSON from this query is below, collapsed because it’s a big block of text. If you look at the name field for each facet, it starts with the raw user agent string we received and ends with the browser we mapped it to.

Summary
{
	"facets": [{
		"name": ["Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko", "IE"],
		"results": [{
			"count": 106
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko", "IE"],
		"results": [{
			"count": 31
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko", "IE"],
		"results": [{
			"count": 14
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko", "IE"],
		"results": [{
			"count": 10
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36", "Chrome"],
		"results": [{
			"count": 5
		}]
	}, {
		"name": ["Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) wkhtmltopdf Safari/534.34", "Safari"],
		"results": [{
			"count": 3
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; FSTE; rv:11.0) like Gecko", "IE"],
		"results": [{
			"count": 2
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:62.0) Gecko/20100101 Firefox/62.0", "Firefox"],
		"results": [{
			"count": 2
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36 Edg/84.0.522.49", "Microsoft Edge"],
		"results": [{
			"count": 1
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36 Edg/84.0.522.52", "Microsoft Edge"],
		"results": [{
			"count": 1
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36", "Chrome"],
		"results": [{
			"count": 1
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36 Edg/84.0.522.44", "Microsoft Edge"],
		"results": [{
			"count": 1
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0", "Firefox"],
		"results": [{
			"count": 1
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36", "Chrome"],
		"results": [{
			"count": 1
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36", "Chrome"],
		"results": [{
			"count": 1
		}]
	}, {
		"name": ["Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko", "IE"],
		"results": [{
			"count": 1
		}]
	}],
	"totalResult": {
		"results": [{
			"count": 181
		}]
	},
	"unknownGroup": {
		"results": [{
			"count": 0
		}]
	}
}
2 Likes

So it would seem that my question still stands - why is NewRelic getting the browser types so wrong on our installation?

I sound like a broken record here - but I downloaded our manual log files for a day, and counted how many IE browser records there were (I searched for the Trident string), and it was like 3% of the total, and when I checked for Chrome type signatures - it was 90% of the total. Again - all in the manual log file I downloaded. And it confirms the type of ratio we have empirically understood.

Now - when I try the queries suggested here on our data using NR (e.g. SELECT uniqueCount(session) FROM PageView SINCE TODAY FACET userAgentName, userAgentVersion, userAgentOS )

  • it reports it the other way around 86% IE, 8% Safari Linux (huh?), and 3% Chrome.

This is just wrong - and I don’t understand why?

It has been identified that we are using the Pro APM browser script on a Lite subscription (I inherited it this way and can only assume a trial was initiated and something not put back), but I’ve been told that this should really only affect data older than 1 day - the current Today data should be correct.

Of course I am trying to see how safe it is to change that APM setting back to Lite to see if this might help (my worry being that it messes something up on a live site - and potentially is not a tick box I can revert if it causes any accidental issue).

But my worry is that maybe NewRelic is just not reliable for a heritage php application, and maybe we shouldn’t be trying to make decisions based on the data it produces? I don’t really want to waste my time (or the helpful responders either to be honest).

Tim

Perhaps I’m misunderstanding. You mentioned there are some “Chrome type signatures” that you’re seeing in your logs that you think aren’t being correctly interpreted. If you can provide me with some specific examples I can do some cross-checking. So far everything I’ve pulled from our consumer has accurately reflected the results from other user agent string parsing tools with the exception of a single record, in which our agent may have had trouble correctly assigning a view from a webkit based browser on Linux.

And again, I want to emphasize that the lite agent vs. the Pro agent should make absolutely no difference. They both obtain and interpret the user agent data in exactly the same way.

Hi - is there a convenient way to privately send you this data (I can remove user identifying info of course)? I can send an excel with the examples I used, and the data that NR reported (not quite exact, as I didn’t snapshot the results, and If I do have a lite subscription then that will be lost from last week - but at least we have what I wrote in my message).

As you say, I am not totally convinced that this is a Lite vs PRo thing - but something else. Possibly something unique to my account, or maybe something to do with the PHP agent or something else?

Tim

Hey @tim.mackinnon1 - let me get you set up to send that info over. Be on the lookout for an email from us.

Hi @hross, I didn’t seem to get a separate email from you/NR however yesterday I did get a support email to arrange a review of our installation to ensure we are setup correctly.

As its turned out - I think we can explain the data discrepancy from that call as we have found that the NewRelic script on our main site, while being included, it isn’t creating a newrelic object in Chrome browsers (but is in IE, and presumably the other specific browser versions for which I am getting data). As one of our other applications is creating this newrelic object, it must be something we are doing in our app that is breaking something.

If you have seen this before, and have any tips, I would appreciate it - and I am getting our team to investigate it - but for now - I think this might be the better explanation vs. an error in the userAgent parsing. And I apologise for any time wasted on this front - unfortunately I have’t yet got enough NR knowledge to diagnose these things myself - however selfishly (by accident) it has helped us find a hidden issue that had gone missed for possibly quite a long time.

Tim

1 Like

Glad you got connected Tim and have some better clues now. I’ll dig around to see if I can find something that relates to what you’re seeing.

Do not apologize for any wasted time - it can take a while to learn what questions to ask, and that’s what we are here for!