Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Feature Idea: Run NRQL Queries against external services, Manta


#1

Hi @hross

Yes please! I think any parameter that is in the GUI should be usable from NRQL, I’m not sure this can be alerted on in a simple way right now, for this purpose I’m using the APM name field to narrow it down on the calls that use Manta and taking the externalDuration for the job. This is not maintainable however as more features are added, I’d very much appreciate this feature.

Many thanks,

Andrew


New Relic edit

  • I want this, too
  • I have more info to share (reply below)
  • I have a solution for this

0 voters

We take feature ideas seriously and our product managers review every one when plotting their roadmaps. However, there is no guarantee this feature will be implemented. This post ensures the idea is put on the table and discussed though. So please vote and share your extra details with our team.


Can we run nrql queries on external services for an app?
Get Error % for an application using NRQL and use it in DataApp Dashboard
#2

Thanks Andrew! I moved your post to a new topic in the Feature Ideas category. Please feel free to add any other details or edit the title or text as you need to. Thanks for your understanding and for sharing with us!


#3

Hi @hross,

I’m filling in a product appraisal comparing New Relic to AppDynamics, I just wanted to say that the point of this issue has been a constant irritant in the process since the NR sales team seems to repeatedly claim that every value in New Relic can be alerted on.

At the moment in my scoring, I’m constantly going against something I know, and that I do actually like, but know is limited when the sales team is claiming it’s not. I’m meant to be scoring this against something that is unknown and for all I know could have a similar deficiency. All I know is what the sales people in NR are saying is categorically false, which as I say is really irritating.

Do they know about this issue? Is there any estimate on when all data will in fact be available in NRQL?

Cheers,

Andrew


#4

@AndrewWalker - Getting more and more metrics converted into event based data (NRQL Queryable) is definitely on our product team’s radar. That said I don’t have a timeline for that.

You may not need these data points as events in NRQL to alert on them however. Is there a particular external service metric you are looking for? Perhaps you can use APM Metric alerting on that?


#5

I wanted to narrow data down to a single external service, Manta. Which I no longer need for the moment.

That’s not the point at this stage though, I know it’s not possible and your sales people are lying to my colleagues.

Cheers,

Andrew


#6

Understood! I totally get how that can be frustrating.

It’s likely that there is just a misunderstanding around what data is automatically collected by default, and what is possible to custom instrument. I’ll reach out to your account team to clarify that.

Please feel free to DM me if you have any further feedback. I’m happy to chat about that outside of this thread :slight_smile:


#7

We’re also very interested in querying the external Services data from APM in Insights. We work with external unreliable APIs a lot. Instrumenting metrics ourselves to get have charts and alerting is cumbersome and a lot of work.


#8

Andrew,

Thank you for your post and for raising this question and concern. I don’t think anyone is deliberately misrepresenting the capabilities of New Relic but a lack of detail in how it would be done can seem that way.

Both you and Ryan make a good point, NRQL is very flexible and NRQL Alerts can be used to drive alerts on any attribute. But Insights events, by default, do not contain every attribute about a transaction event that you see in the UI; eventually they should. Attributes about specific external service calls are not automatically included in the default Transaction event. They can be added to those events, or a specific External Service event type can be created, using custom instrumentation through a configuration file or the Agent API. For example, the external service name and response time can be added for the "Manta" external service can be added as attributes of the Transaction event.

External services calls are an alert-able category of both the standard metric and metric-baseline alerts. You can define alerts directly on the External Service call metric based on the service name through the Alert definition drop-down list. I believe you are familiar with the Alert definition screen but in case, here is an example screenshot for where you would configure an external services alert in New Relic Alerts,

The topic of your post is important and we agree. As Ryan said our product direction is to move more data into Insights. This work is ongoing as we update and release features and products. You can see this with the new event types that have been introduced over the last couple of years like the Span events which are generated by our Distributed Tracing capability, ErrorTrace events which come with our Error Analytics feature, and the extensive set of events available with our Infrastructure agents.

We would be open to a call or web meeting to provide more specifics for setting up the alerts on your external services or for enabling additional attributes for external services on transaction events. There is guidance and resources that we provide to help customers with extending the standard attributes and events. Please let us know if you would like to discuss this further.

And, again, thanks for your suggestions and comments.


Get Error % for an application using NRQL and use it in DataApp Dashboard
#9

The thing is that each transaction could connect with multiple external services. The ideal situation here would be a join between each transaction to an external service transaction table on a transaction ID, but you can’t join.

I’d imagine the join limitation is holding back this feature and others. But that’s for another more well-known feature request thread. And if someone suggests selecting from multiple sources please don’t, there’s no method to set an identifier on each source to filter them independently and you can’t join to the same table in any useful way either. You also can’t join on any particular attribute.

Ok, I’m sorry if the above seems very negative, I do appreciate you trying to help. I’ll try and reciprocate by giving an example, say you have a single service that is sporadically performing very badly and is logging transaction into the transaction table, how can you define a search on transactions that are above a certain APDEX score for that single service, not every single service, and get an average external service transaction time for each of the external services associated with that external transaction type. Alternatively, perhaps for each bucket of 0.05 on the Apdex scale you could chart it on a line graph from 0 to 1 on the X axis, and average time on the Y axis, with each line representing the average duration for a transaction that was taking a long time.

Even with the method above, you couldn’t do anything like this. I don’t know whether any of your competitors can do it either, but you give a developer, or a DevOps engineer, or a DBA access to a SQL like interface and there are certain things they expect to have, access to all data, common SQL functions like join, mod, left, right, etc then they should be able to use having, to be able to work with timestamps and durations with ease, and I could go on.

Thank you for pointing me to that alert condition, it does address part of the concern I had, however, it’s not the whole picture.


#10

Andrew,

It’s clear that you have a lot of knowledge of NRQL. It is not a full implementation of SQL and capabilities like JOIN are not implemented in NRQL. And thank you for providing the example. I see how that particular use case would be challenging but I would like to review it with some of our engineers to see if we can suggest some alternatives to a query-based solution.

If you don’t mind I will follow-up with you through DM.


#11

I wanted to see percentile data for external services response time, currently the monitoring view only shows average and throughput

I thought I could maybe create a custom dashboard via Insights, but it turns out I can’t even query external services at all :frowning:


#12

Hey @paul_dp You’re right, for now External Services aren’t in Insights by default.

With that said, we are trying to make more and more data queryable, by making a lot more data Event-based, rather than metric. So I hope that soon enough almost all data will be available in Insights.

I don’t have a timeline on that, this could be still a long term plan. So I’ll get your +1 added such that we can let the PM team know about the great interest in this. :slight_smile: