Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Feature Idea: Remove 1000 limitation for Insight NRQL results

nrql
feature-idea

#1

Please paste the permalink to the page in question below:

Please include any NRQL you are using:

select * from pageview limit 1000000;

Please share your question/describe your issue below. Include any screenshots that may help us understand your question:

The max value of limit is 1000. but i want to get 1000000 each query. Are there any approach to unlock it?
thanks,


New Relic edit

  • I want this, too
  • I have more info to share (reply below)
  • I have a solution for this

0 voters


Whats the limit on number of results returned for NRQL?
Feature Idea: More Flexibility in NRQL Results Display
Feature Idea: How to increase the limit value in the nrql query
#2

@kelin.liu Thanks for posting to the forum. I’d be more than happy to create a feature request on your behalf to increate the query limit beyond the current 1000 limit. Although I can’t guarantee that our developers will move forward on this functionality, I’ll be sure to link this forum thread to the internal tool we use to track our feature requests.


#3

Was a feature ticket ever created?

This would be very useful for our organization as well.

Thanks.


#4

Hi @sams,

This feature request is currently being tracked in our internal system. I have added you to the feature request to let our product managers know about your interest in this functionality. Thanks for sharing your interest


#5

Can you please add me as well… The limit of 1000 is very limiting considering the amount of data we have out in NR.


#6

I’m using the Insights API to ‘select * from MobileSession’ but I only get about 4 minutes worth of results. Is there a limit on the output limit from the API?


#7

Queries are limit to 1,000 results each.


#8

To get your maximum 1,000 results, you may need to add “LIMIT 1000” to your query - without that, query results are limited to 10 and/or add “SINCE [some time frame of more than 1 hour]” - without that, query results are limited to the past hour.


#9

can we unlock 1000 limitation of each result?

thanks,


#10

We don’t currently offer large limits, but I’ve gone ahead and passed your feedback on to our product team. If this limit is increased we’ll reach out to let you know. Thanks!


#11

Any update on this? Has Max Limit been increased?


#12

If the limit cannot be increased platform wide, it may be beneficial to increase the limit on the queries used to populate the Infrastructure UI. For example, servers using a specific version of a package on the Inventory screen.


#13

This is definitely one of the more puzzling and disappointing limitations of New Relic. Other industry products provide result sets in the millions (or more). The 1000 result set limitation makes some necessary queries functionally impossible.


#14

Considering the value that lies in the data captured by New Relic it is quite a set back not being able to run some deeper analysis on it. In many cases the interesting patterns only become visible with more complex queries using for example subqueries or joins. I can (somewhat) understand that such complexity is (not yet) surfaced within NRQL since it can also cause some unpredicted load on the systems if not used wisely. Thats why we need to be able to export the data into our own data warehouse, do some additional analysis and push the results back into NR for visualization. With a limit of 1000 records and no good option to use pagination in order to pull say all the pageviews of our application we tend to look at other solutions which offer such flexibility. If REST or GraphQL do not offer this having another approach to export the raw data within a table as we currently have to do it using SELECT * FROM would also be a good solution.
Without this we sadly have to recommend other tools which can do this to our customers.


#15

I think the best solution would be to implement a pagination feature like the one recommended here:

This effectively removes the 1000 limit without requiring a massive response size. Barring this solution I am in full support of removing the limit since the status quo provides no way to handle a large dataset.

EDIT: I’ve just discovered the OFFSET keyword in NRQL which, in conjunction with pagination, seems to open the doors to our dataset
EDIT 2: It turns out the OFFSET+LIMIT must be less than 1000 which is very, very disappointing. Without the ability to page through our data Insights does not fit our needs


#16

Curious as to the status of this feature request, as I have a somewhat more fringe usage case.

I attempting some bulk data analysis, using some in-house data, and using the API to pull related data out of New Relic. Originally, I was trying to pull a 24 hour set of data, that I could parse through on my local machine (working from a BASH script). The problem is that 24 hours of New Relic data is typically in excess of 3,000,000 records. Obviously, that can’t currently be done.

I was able to rework my logic, so that I am only pulling a handful of records at a time, but now, rather than one large pull up front, I am doing hundreds of thousands of pulls. The result of having to wait on the API every time is that processing 24 hours of data is currently taking me over two weeks. (wholly unacceptable)

If I could pull all my data in one shot, even if it took an hour, it would only take my script four or five hours to parse though the local copy.

I understand the logic behind limiting the length of the returned data, but an override would be nice, for the odd times when someone needs to do some heavy lifting. (or maybe a separate API for bulk pulls?)


#17

Hi, @jfornango: Are you working with a New Relic account team (account exec, customer success manager, etc.)? If so, I recommend that you describe your use case to them; they may be able to offer an alternative.