Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Pearson Coefficient for Response Time vs Throughput

nrql
statistics

#1

Kevin Scaldeferri put together an article named “Secrets of a NRQL Wizard” that I refer to regularly for an advanced NRQL refresher.

Article https://blog.newrelic.com/product-news/advanced-nrql/

At the bottom the article, there is a discussion of Pearson Correlation Coefficient.

Formula:
( count() * sum(x * y) - sum(x) * sum(y) )/ ( sqrt(count() * sum(pow(x,2)) - pow(sum(x),2)) * sqrt(count(*) * sum(pow(y,2)) - pow(sum(y,2))) )

His example uses databaseCallCount and duration.

I want to correlation response time (duration) and external service calls (externalCallCount). I haven’t gotten it to work yet.

Here’s my NRQL:
SELECT
SELECT (count() * sum(duration * externalCallCount) - sum(duration) * sum(externalCallCount)) / (sqrt(count() * sum(pow(duration,2)) - pow(sum(duration),2)) * sqrt(count(*) * sum(pow(externalCallCount,2)) - pow(sum(externalCallCount,2)))) AS ‘pearson correlation’ FROM Transaction WHERE appName=’’ AND externalCallCount IS NOT NULL

When executed I get: NRQL Syntax Error
Not supported: Call{target=Id{name=‘sum’, isQuoted=false}, arguments=CallArguments{positionalArgs=[Id{name=‘duration’, isQuoted=false}], kwargs={}}} (at line 1 column 141)

Has anyone else successfully gotten the Pearson Correlation Coefficient formula to work?


#2

That’s an interesting one @CHRISTOPHER.MCKENNA - I have not been able to get that query to work either.

Let’s ping Kevin to see if he could shed some light on that query :slight_smile:
@kscaldeferri


#3

Hi @CHRISTOPHER.MCKENNA,

It looks like there’s a small typo in your formula. pow(sum(externalCallCount,2)) should be pow(sum(externalCallCount),2). I was able to get
SELECT (count(*) * sum(duration * externalCallCount) - sum(duration) * sum(externalCallCount)) / (sqrt(count(*) * sum(pow(duration,2)) - pow(sum(duration),2)) * sqrt(count(*) * sum(pow(externalCallCount,2)) - pow(sum(externalCallCount),2))) AS 'pearson correlation' FROM Transaction WHERE appName='...' AND externalCallCount IS NOT NULL

to work for me. (For other readers, note that in the original post, count(*) is being rendered incorrectly in a couple places.)

Glad to hear that you’ve gotten value from the post :grin:


#4

Thank you for your response @kscaldeferri . One parenthesis can throw the whole thing off.

I’m curious, can I use the coefficient to correlate throughput and response time using NRQL?