Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Relic Solution: Querying Insights usage across time

developer

#1

What is the recommended maximum date range to be able to execute considering the attributes related to “sum(insightsTotalEventCount)-sum(insightsIncludedEventCount)” are not accurate for various days along with sum of total events during date period.

I recently received a question about querying Insights usage across time. We’re pretty upfront that insightsIncludedEventCount and insightsTotalEventCount may not return what you would intuitively expect.

Note: This attribute represents a count of events existing in storage as measured on a given day, and not a count of the events added to storage during a given day. Summing this attribute over a period of more than a day may not give you what you expect; for example, it will not yield the total volume of events written to storage in that period.
Source: Insights attributes

I wanted to go over how this data is generated, how we actually bill for this data, and how to query for it over time.

Data generation

Insights usage data, like other usage data, is calculated once every 24 hours at about 9:15 UTC. This is important to remember when querying for usage data. The timestamp on any NrDailyUsage event will reflect the day and date of the usage, but it will not show you a more granular temporal resolution.

As you can see, it’s a pretty spikey chart. I would encourage you to keep this in mind when working with usage data.

How we calculate Insights usage

This is where it can get a little complicated. Insights is billed by the average daily total of stored events not covered by other retention policies. I’ll cover these in reverse order.

Not covered by other retention policies

This means that if you’re storing APM Transaction events on a Pro subscription, the first 8 days don’t count towards your Insights usage. You can see this in many official usage queries as:

sum(insightsTotalEventCount - insightsIncludedEventcount)

Subtract the number of events covered by other subscriptions from the number of total stored events to get only the number of events stored under the Insights subscription. We use sum because there are distinct usage events for each Insights Namespace i.e. APM Transactions, Mobile Requests, etc. We sum all of the different namespace totals and included totals for each day.

Daily total of stored events

This means we only care about the total. The number that you see next to insightsTotalEventCount is not the number of events added that day. Events will fall off the end of their retention period on a regular basis. The NrDailyUsage event does not directly account for how many events are added or removed in a given day, only the stored total. If you add new events every day, it’s not uncommon to see a very flat graph. This is a tricky part of where rate() and TIMESERIES comes in to queries.

rate(sum(insightsTotalEventCount - insightsIncludedEventCount), 1 day)

This is the same NRQL function that we use in the queries in the Usage UI. However, like the event counts, it’s not the whole picture on its own. Rate will return the rate per day of Insights usage

Rate without a TIMESERIES or FACET clause will return the per unit rate for the time range covered in the search. This doesn’t play well with a daily total. I highly recommend making sure you set the TIMESERIES clause to 1 day. This buckets the data per day. Consider the following queries:

You may notice very different results. The second chart is an accurate depiction of the daily total stored events, the first chart spikes up to 8 million! This is the result of the automatic grouping in TIMESERIES

1 day buckets - Chart 2

350, 000 events are generated at 9:15. No other events are generated outside of the time, so we have a rate of 350, 000 events per day.

1 hour buckets - Chart 1

The 1st chart is defaulting to TIMESERIES 1 hour for a query covering a range of 3 days. This gives some extremely spike day rates.

8:00 - 9:00

0 events are added. 0 events per hour * 24 hours in a day = 0 events per day

9:00 - 10:00

350,000 events are added. 350, 000 events per hour * 24 hours in a day = 8.4 million events per day

10:00 - 11:00

0 events are added. 0 events per hour * 24 hours in a day = 0 events per day

Average

Finally, we come to the average. I will continue to reiterate that NrDailyUsage events record the total stored events for that day rather than the number of events that have been added. This means we want the mean value and not the sum. We do, however, need to sum the totals across Insights Namespace per day. This means that most Insights usage queries across time will want to have:

rate(sum(insightsTotalEventCount - InsightsIncludedEventCount), 1 day) … TIMESERIES 1 day

Hopefully, this gives you enough information to get an idea of what to expect when summing over a period of more than a day. When in doubt, the Unified Usage UI is the source of truth for your usage.