Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Relic Solution: Data visualization and bucketing in the New Relic Infrastructure UI

levelup
newrelic-infra
infrastructure
visualisation
bucket
rfb

#1

Why do my graphs change shape?

In the Infrastructure dashboard, you might notice that your graph changes shape from moment to moment (on a very short time window) or when moving from a short time window to a longer one. I’m here to help you understand a little better about how data is bucketed and aggregated in Infrastructure. This should help you be able to better interpret the graphs that are generated. I’ll do this by using CPU % as an example, but the principles hold true no matter what metric you’re viewing in a graph in your Infrastructure dashboard.

5-minute graph:

Here are two screenshots of a 5-minute timewindow, taken about 5 seconds apart:

Notice that the graph changes shape. In another 5 seconds, it goes back to looking more like the first graph, and continues to cycle through these shapes (while also adding more recent data to the right side). Why is this?

The reason is due to bucketing. Infrastructure collects CPU metrics once every 5 seconds from your host. However, the 5-minute time window displays 10-second buckets. This means that two 5-second data points are being aggregated to display each data point on the graph. You can hover your cursor along the primary line in the graph to see the time stamps of each data point, noting that they are 10 seconds apart.

When the dashboard receives another 5-second data point, it pushes all the data points one step to the left to make room for it. This moves every other 5-second data point into its neighboring 10-second bucket. Since each 10-second bucket is actually an aggregation of two 5-second raw data points, the aggregation will result in a slightly different number, and the data point on the graph will appear to change. The raw data is never changing, but because it’s being moved from bucket to bucket, the graph will alternate between two shapes as the raw data points get shuffled from bucket to bucket.

6-hour graph:

Here is the same server as seen on a 6-hour timewindow:

You will notice that all of the spikes seen on the 5-minute graphs above have flattened out. This is also due to bucketing: a 6-hour time window uses larger buckets than a 5-minute time window. The aggregation of many data points results in a much smoother line. A 6-hour time window uses a 15-minute bucket size. What this means is that, when comparing a 5-minute graph to a 6-hour graph, you have to keep in mind that each data point on a 6-hour graph is actually 90 buckets that have been aggregated together.

Interesting! What other bucket sizes will I encounter?

The default time windows in Infrastructure use the following bucket sizes:

  • 5-minute time window: 10-second bucket size
  • 30-minute time window: 1-minute bucket size
  • 60-minute time window: 1-minute bucket size
  • 6-hour time window: 15-minute bucket size
  • 24-hour time window: 30-minute bucket size
  • 7-day time window: 6-hour bucket size

But my graph flattened out and I didn’t change the time window!

There is one other detail that can affect the shape of your graph in Infrastructure. Graphs will auto-scale based on the range of data seen. Notice up above, in the first screen shots I posted, the graph only goes up to 2.5%. This is because the data stream is never going above ~2.3%, so the graph auto-scales to show as much detail as possible. However, if my CPU % experienced a sudden spike of, for example, 60%, the graph’s scale would immediately change to accommodate this new data point. The autoscaling would cause all of the old spiky data points to flatten dramatically, so that the large spike at the far end of the graph would fit on the graph. Note that the raw data never changes, just the scale of the graph showing the data.

I hope this helps you to better understand what you’re seeing in the Infrastructure UI, and why your graphs might change shape unexpectedly. Happy monitoring!


Relic Solution: Downsampling and Retention of Infrastructure Data
#2

Data aggregation is always tough when people are visualizing data. How do graphs handle gaps in data? Is there a blank or other indicator that the data was missed or does the graph simply compensate and display a contiguous line?


#3

It really depends on the bucket size you’re currently viewing.

However, if one or more buckets-worth of data are missing (usually do to the agent losing connection or the server going down), the graph will “flatline,” showing a steady 0 for all metrics.

If the server of interest was also attached to a Host Not Reporting alert condition and the data gap lasted longer than 5 minutes (or whatever the threshold is set to in the condition), a Host Not Reporting violation would also be opened and the graph would turn pink in that region.


#4

Nice! One more reason to make sure that Host Not Reporting alert is configured. Thanks for the quick response.