Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

How to get an alert when, "No data reporting for this application"?

apm
insights
feature-idea
rfb

#1

Good day,
I’m beginner to new relic, I faced some issues while setting up monitor of an application is up and running or not and to alert when not running. Below is my scenario:

When an application has been stopped in server, the agent will be stopped reporting data and it will be visible in NewRelic > APM > Application as “No data reporting for this application”.

In an another discussion, I found that to alert for this purpose from insights we could use the following query, “SELECT count(*) FROM Transaction WHERE appName = ‘[your application name]’”

One of my application will just take the request from user and fetches data from third party, so for this situation the count may result 0 this doesn’t mean agent stopped reporting whereas user is not accessing the application. If I use the above mentioned query it may provide a fake alert to team and cause a discrepancy.

Any suggestion how to overcome it ? How does NewRelic predict “No data reporting for this application”. NewRelic shows as “greyed area” but I want that to alert when agent actually stopped reporting data to application.

Thanks in advance.


#2

Hi Shriram,

We currently don’t have a application not reporting alert at this time. The alerts system does need data in order to evaluate. I’m not sure that your query on transaction count will work here.

Regards,

Paul


#3

Hi @Shriram.Krishnan - If I have read your question correctly, you want to know if a call to a 3rd party is unavailable? If that is correct then you could write an API Synthetic to call the 3rd party endpoint


#4

Hi Stefan,

Thank you for your time.

Concern is not about third party endpoint is available or not. I have a component which built to call the third party. My component should be up and running to call third party. The call to third party will be applicable only when user request in application.

Here I want to check my component / service is up and running or not. My component in New Relic APM agent will collect data when my component runs. If my component is not running / stopped then APM agent will not receive data. So in UI it will show as “No data reporting for this application”. I just want to turn that as an alert when an application says, “No data reporting for this application”.


#5

Hi,

Have you managed to sort out this as i also have same situation where i just want to know that my application is running and connected to new relic agent.

Regards,


#6

Ways to implement your requirement.

  1. Develop your custom script send the feed to Insights and make Newrelic to alert. When the custom script detected that something wrong obviously it will send to Newrelic.
  2. In NewRelic Infrastructure its possible, Pre-requisite : Infra agent should be installed in your host server where the application running. Newrelic has the capability to capture the process command line argument. If the specific application command line argument not running. Then application wouldn’t be able to connect to New relic. So this inturn proves application not running.

But still I’m looking foward from NewRelic team to have this feature built. When APM has no data collected for some “X” min then it should have alert mechanism.


#7

There is a solution to this which is a NRQL alert condition. I have provided an example on a previous post:


#8

This wont work if User is not using a system since there will be no transaction in that case.


#9

@stefan_garnham It wont work as I stated in above thread. Because your solution makes user to use the application all the time. But its not possible based on design for some application. I suppose.

Between, hope you are doing well.


#10

It sounds like you’re wanting a kind of heartbeat condition where the agent will tell New Relic that it’s still running even when no real transactions are coming through. Does that sound about right? I can totally understand the desire for this sort of monitoring and it’s something that other folks have requested too. Monitoring low throughput transactions and applications is a pretty complicated problem to generically solve!

One solution my team has used is to have a Synthetics monitor regularly hit a couple endpoints for our service’s API to keep some level of traffic coming through all the time. You can then configure a condition like @stefan_garnham suggested. If Synthetics doesn’t work as a solution for you because of architectural concerns or the service not being accessible publicly, you could even setup a cron job that would curl that endpoint periodically (once a minute works but if it’s just a curl you could even do it more often).

Another indirect monitoring solution my team uses is to monitor the error rate of something that has the service in question as a dependancy. We are responsible for the uptime of a service that is critical to consumers of our team’s services (Service A) that has some unique monitoring challenges. We settled on having the Infrastructure agent running with a Host Not Reporting condition, so we could be sure that the machine this service runs on is healthy, and we combine that with a condition targeting the error rate of a service (Service B) that we know will get a very high error rate if Service A falls over or stops responding to requests.

Other details for this are notes in our runbooks to verify that Service A is reachable when responding to pages before trying to dig too deep. The idea for all of this is that if you get paged in the middle of the night, you’ll open the runbook link included in the notification and start following those instructions. An engineer can be woke up and while their brain is trying to get a handle on what’s happening, they have a couple of quick checks to make sure the most critical pieces of our team’s infrastructure are working. If they aren’t well fire up the coffee and get to work getting them online. If they are then maybe you don’t have to shake off your sleep quite so quickly and it’s something that could get some further examination in the morning.

I realize that direct monitoring solutions are preferred, and I always strive to use them when possible. Low throughput applications and transactions are a challenging case. Right now the agents aren’t really designed in a way that makes the sort of heartbeat monitoring I think you might be asking for super easy. If that is what you’d like, let’s make sure we get this filed as a #feature-ideas:feature-ideas-alerts so that your voice can be heard. I promise that this is something you want to speak up about! It makes a difference when teams are weighing feature work and the Alerts team especially takes customer feedback very seriously. (Everyone else does too, I just have a soft spot for Alerts :smiley: )


#11

+1 for #feature-ideas:feature-ideas-alerts


#12

One thing we’ve done with some of our custom on-host integrations is to send up a “heartbeat” event every x minutes. This event is separate from any transaction or user induced metric. It’s our way making sure the integration is still running, as we can write NRQL conditions that check for this event. If there isn’t an event in the interval window, then we fire an alert. We’ve found that sending this event on a fairly spacious interval (e.g. every 10 minutes or so) doesn’t eat into our Insights limit that much.

The catch with this approach (at least in our case) is that in order to effectively monitor you need to know how many hosts the heartbeat transaction is coming from so that you can tune your NRQL query properly. For example, if we have our integration running on two hosts, then we’d expect to see at a minimum two heartbeats in interval window. As we scale or shrink hosts, our NRQL has to change as well.


#13

This is a great approach, too, especially with custom integrations. Depending on your architecture you could use AWS Lambda (or if you’re not heavily in AWS build in a step to your scaling process) to change the condition to reflect your new number of hosts. I could also imagine this being a good candidate for FACET support but, again, that would depend on how you’ve got things setup.

@chad.hays I would actually love it if you were feeling adventurous and wanted to try out FACET for these heartbeat events. You might be able to remove a step from your scaling process.


#14

+1 for #feature-ideas:feature-ideas-alerts