Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Custom metric values do not agree with app (are very large)


#1

###Insights Question Template

  • There are many ways to achieve a goal in Insights, tell us a bit about your end goal. What insights are you hoping to gain?

We are trying to track a time duration as a custom metric using the Python agent. This is integrated, but the values that the Insights dashboard displays do not agree with what we are passing it. They are 3 orders of magnitude larger.

It may be that the agent is mis-configured, but at this point I’ve verified that we are passing it durations in an integer representation of milliseconds, and that the integer appears to be sane.

The min value view has very reasonable numbers. Is there some kind of aggregation going on that I’m unaware of?

  • Please include the URL to the Insights dashboard that needs addressing:

https://insights.newrelic.com/accounts/788482/explorer/metrics#entityId=172652160&entityName=statistics-reviewsearchfiller-production&searchTerms=time&entityType=application&entityTypeName=Applications&view=METRICS_OF_ENTITY&timeWindow=6&endTime=1546961160000&metricAgentIds[]=172652160&metricNames[]=Custom%2FReplication%2FTotalTime&metricFacets[]=false&metricSegments[]=Custom%2FReplication%2FTotalTime&metricFuncs[]=average_value


#2

@nea - Based on the informations I’m seeing going on “under the covers” of your custom metric, the values being displayed in the dashboard you linked appear to be totally in line with the data as it is being sent. Here is a snippet of the raw data values, as was recorded by our backend as emitted from your agent.

Start_time      End_time       Count                Duration            Exclusive_time       Min    Max          Sum_of_squares  
Today, 10:31	Today, 10:32	267.5833333333333	4,487,596,317,081.6	4,487,596,317,081.6	1,647	114,868,994,048	3.534287693824525e+23

The formatting is a bit difficult to represent via the forum, but this can essentially be mapped to the raw data values as recorded by the agent with the record_custom_metric API call. This data represents 1 minute worth of data as sent by your agent, which shows the absolutely huge numbers being submitted by your use of the API call to the agent. When invoking the agent AP, are you recording an individual int just once per minute, or are you passing in a dict and supplying the additional values as seen in the snippet I provided. I’d have to confirm in the Python agent code but I believe if the API call is being made multiple times within the span of a single minute, the agent may automatically aggregate and sum individual int values to a dict automatically. If this is the case, we would still see individual API calls being made with values of 114 billion as represented by the max attribute from the data snippet.

For more information about the various metric value fields, see this documentation on metric_value_fields

You should also be able to enable audit_mode to see the raw data payloads being sent by your Python agent as it is sent to us each minute to confirm what is being recorded by the agent.

I hope this helps you track down the source of your unexpected data values but please don’t hesitate to let us know if you need any further assistance in investigating the source of your unexpected data values.