Querying prometheus counter types in NR

Different results from Prometheus and New Relic

I’m having some trouble translating a simple promql query into nrql. I am using the postgres-exporter to pull data out my postgres instance running in RDS, and I write to New Relic via the prometheus remote write integration.

Anyway, when I try to query the data via NRQL I get way different results from when I try to run queries in prometheus. For example, something simple as


which gets translated in NRQL as

SELECT rate(sum(pg_stat_database_tup_fetched), 1 SECONDS) FROM Metric WHERE (datname = 'buzz') SINCE 60 MINUTES AGO UNTIL NOW FACET dimensions() LIMIT 100 TIMESERIES 60000 SLIDE BY 30000

I’ve attached some screenshots to show you why I am a bit confused.

Any chance your time is different? You could be offset from the two sources.

could be but even with a change in the locale used by the 2 systems, I wouldn’t expect such a difference in the values returned.
Look at the results given by the promql running in prometheus, we go from 0 to almost 200. In the graphs in New Relic we have a horizontal line that sits on the 3.5M.

Doing some more testing and it seems that what prometheus identifies as a counter is translated into a gauge in NewRelic, or at least this is what I understand. I’ve attached screenshots to show the difference

From the exporter /metrics endpoint:

# HELP pg_stat_database_tup_fetched Number of rows fetched by queries in this database
# TYPE pg_stat_database_tup_fetched counter
pg_stat_database_tup_fetched{datid="00000",datname="foo",displayName="foo",server="foo-postgres:5432"} 5.3130852e+07

And in New Relic:

@valerio.barrila Following up to confirm that your further testing helped you resolved this issue? If so please mark it resolved. Thank you :slight_smile:

@JoiConverse not yet, we received an answer to our support ticket yesterday, we’re going to check in the next couple of days and I’ll post here a solution if we can confirm it works.

1 Like

We found a solution that involves metrics relabeling in Prometheus to make New Relic interpret them correctly.

1 Like