Creating a template data app

I have been researching Google’s SRE methodology, which appears to closely resemble New Relics DCX. One of the problems that my company deals with is reporting consistency.

Here are my thoughts and I hope there’s a solution that someone may be able to suggest for my follow up question.
I am working with several teams to create data apps that have a consistent look and feel to them. I am first attempting to categorize the apps into groups such as “Customer-facing”, “Batching”, “Database”, etc. Each category will have a set to service level indicators that are uniquely important to them. A batching system does not care about millisecond latency, another will care about “data freshness” while others will not.

With that in mind, here are some questions that I hope someone really smart can help me with:

  • Can I create a data app template, with all of the charts I want for a category, and make a copy of it leaving me only to change the appName field in each chart to apply to the app?
  • Is it possible to create a data app and use some sort of filter to superimpose the app I want to see on top of each chart? This would be one data app that would fill the need for all of the apps in a customer facing category, another for batch, another for other categories.
  • (This is more of an observation than a question) NR appears to follow the SRE (DCX) methodology in part, but I have not found template NRQL to report on what I would think are “typical” jump start items to measure. There’s a post that contains some specialized NRQL by users HERE, that helped quite a bit, but I can’t seem to find “Startup” type queries. How to’s with basic “Latency”, “Volume”, “Availability”. My intent is to follow up on this and post my NRQL for these items.
4 Likes

Here are the queries I am using, in the data app I am able to separate them by tabs.

OVERVIEW TAB

On this tab, I use a dashboard note to clearly identify the objectives as well as define the Service Level Indicators and Service Level Objectives.

Service Level Indicators and Service Level Objectives

Service Level Indicator Details Service Level Objective
Latency Response time < 400 ms or < 850 ms — 90% < 400ms and 99.9% < 850ms over 30
Availability All non-500 response codes — 99.9% over 30 days
Throughput Maintain latency SLO when throughput is below 9,000 rpm — 98% over 30
  • Percent of Responses less than 850ms (Are we meeting the objective set?)

SELECT percentage(count(*), WHERE duration <= 0.85) FROM Transaction WHERE appName = ‘XYZ App’ since 30 days ago

SELECT percentage(count(*), WHERE duration <= 0.4) FROM Transaction WHERE appName = ‘XYZ App’ since 30 days ago

I then use threshold settings on the chart to indicate issues.

LATENCY TAB

  • Tiered Response Times

Detail on response distribution

SELECT filter(average(duration), where appName = ‘XYZ App’ and duration <= .40 ) as ‘Best Response < 400ms’, filter(average(duration), where appName = ‘XYZ App’ and duration <= .80 ) as ‘Good Response < 800ms’, filter(average(duration), where appName = ‘XYZ App’ and duration >= .80 ) as ‘Poor Response over 800ms’ FROM Transaction TIMESERIES AUTO

Average Response Time
I don’t like averages as they tend to soften the painful moments, but here a query for that.

SELECT average(duration) from Transaction where appName = ‘XYZ App’ TIMESERIES Auto

VOLUME TAB

Total Volume indicator

SELECT count(*) AS ‘Requests’ FROM Transaction WHERE appName = ‘XYZ App’ TIMESERIES AUTO

What is the busiest day

SELECT count(*) FROM Transaction WHERE appName = ‘XYZ App’ SINCE 1 week ago FACET weekdayOf(timestamp)

Total volume for the month, also on the overview tab

SELECT count(*) FROM Transaction WHERE appName = ‘XYZ App’ SINCE 30 DAYS AGO

Total volume over the last hour

SELECT count(*) FROM Transaction WHERE appName = ‘XYZ App’ SINCE 60 MINUTES AGO COMPARE WITH 1 WEEK AGO TIMESERIES

DATABASE TAB

  • CONSUMPTION

The average amount of the time used by the DB in a transaction.

SELECT (1-(average(duration)-average(databaseDuration))/average(duration))*100 AS ‘% DB Time Consumed’ FROM Transaction FACET name LIMIT 5 WHERE appName = ‘XYZ App’

The number of DB transactions over a threshold

SELECT count(*) FROM Transaction WHERE databaseDuration > .2 WHERE appName = ‘XYZ App’ TIMESERIES AUTO

A percentage view of transactions over a threshold (I used this in an attempt to correlate long DB times with long transaction times.

SELECT percentage(count(*), where duration >= .5) from Transaction where appName = ‘XYZ App’ TIMESERIES AUTO

SELECT average(databaseDuration) FROM Transaction WHERE appName = ‘XYZ App’ TIMESERIES AUTO

Percentage of requests over the desired DB response threshold

SELECT percentage(count(*), WHERE databaseDuration <= .2) FROM Transaction WHERE appName = ‘XYZ App’ Since 30 days ago

This is a last hour visual of slow DB queries

SELECT percentage(COUNT(*), WHERE databaseDuration > .2) FROM Transaction WHERE appName = ‘XYZ App’ TIMESERIES AUTO

AVAILABILITY TAB

Percentage of “Good” responses

SELECT percentage(count(*), WHERE (httpResponseCode LIKE ‘20%’)) AS ‘% HTTP 200 Responses’ FROM Transaction WHERE appName = ‘XYZ App’ since 30 days ago

Percentage of “Bad” responses

SELECT percentage(count(*), WHERE (httpResponseCode LIKE ‘5%’)) AS ‘% Server-side Error Responses’ FROM Transaction WHERE appName = ‘XYZ App’ since 30 days ago

Listen, I hope this is helpful for some folks to jump in and get started. I can say that I really recommend taking a look at the SRE book that Google put out. https://landing.google.com/sre/sre-book/toc/
It has been a huge help and I would welcome any improvements to the queries I’ve developed in addition to questions you may have about why I did something the way I did. :v:t3:

3 Likes

So I discovered a work around to not being able to make a copy of a data app. I have found that I can make my tabs as dashboards, minus the appName in the query. I can then copy them into a newly created data app, and modify each of the charts to add the appName back into the each NRQL Query.

An alternative option is to create a generic dashboard and then enable to filter option for appName, the issue there is you would have to select the appName each time you change tabs since it’s not possible to enable a filter for the entire Data app that I can find.

Still looking for ideas if anyone has any.

2 Likes

To clarify a bit more on the last comment… When you do a generic board, all the queries above need to have the “where appName = ‘blah’” removed. Select edit on each page and configure the filter options to Transaction, and Appname fields respectively.

Wow! @reopelle.scott, I love this post. Thank you so much for sharing your expertise with the community. I agree that consistency in reporting is hugely important in sustaining the value of the reports.

In case you haven’t seen it yet, we have a great, in-depth, section of our Docs focused on getting started with DevOps called “Measuring DevOps Sucess”. I highly recommend all customers give it a read.

https://docs.newrelic.com/docs/using-new-relic/welcome-new-relic/measure-devops-success/guide-measuring-devops-success


To address your questions:

Can I create a data app template, with all of the charts I want for a category, and make a copy of it leaving me only to change the appName field in each chart to apply to the app?

It sounds like you discovered the solution to this already. The individual dashboards that make up a data app can be copied from edit mode. Those new dashboards can then be imported into any data app.

Is it possible to create a data app and use some sort of filter to superimpose the app I want to see on top of each chart? This would be one data app that would fill the need for all of the apps in a customer facing category, another for batch, another for other categories.

Filters can be enabled for a Data App from “Edit” mode, allowing you to select multiple event types and attributes that can be filtered on by searching in the Smart Filter box (that is only visible after enabling filtering). Filters are applied to all tabs in a data app. You can choose the application as a filter and see it applied to all charts on all tabs. By typing the application name in the Smart Filter, you’ll see an automatically generated list of matching attributes and values you can filter by.

3 Likes

As a follow up, I wanted to link this post for additional context. If you are using Averages, you are missing critical events