Reduce 'Metrics' data costs

Hi All,

We run clusters on GKE, and we are really interested in Nodes/Pods monitoring, APM and Browser features, but we have little use (for now) of time-series about everything. We are trying to figure out how to reduce our monthly costs, most of our data ingestion is from “Metrics”

Can we disable metrics collecting ? If so, we’ve found enable_process_metrics config. Will disable this prevent other integrations ( K8s, or APM ) to work correctly ?

Thanks

1 Like

@sammmmmmmmm I think this doc would be able to provide some guidance: Manage data coming into New Relic

Hi @JoiConverse,

Yes, i’ve found that doc before, but it’s not really relevent to helping understand where the data come from, and how to pick which config to modify in order to enable or disable each data ingress.

As for example, we have a DAILY ingress of :

  • 54.68 GB “Metrics”
  • 15.78 GB “Infrastructure integrations”
  • 4.31 GB “Infrastructure processes”
  • 6.11 GB “Infrastructure hosts”

How can i know what’s actually included in the 55GB a day of Metrics ? That’s almost 39Mb per minute !

Outside office hours, the cluster doesn’t even have that much network traffic!

As for Infrastructure integrations, we only have GKE integration configured. How can we know which part of GKE integration is sending that much data ? The only thing we want to know with the GKE integration is how many pods/services are healthy, and which Deployments doesnt have required pods.

Have a great day!

2 Likes

Thanks @sammmmmmmmm. Will definitely loop in one of our support team members to help! :slight_smile:

@JoiConverse Not sure if i am contacting the wrong people, but i am trying to reach your support on that subject since Day 1, But i never had any answer to my questions. I tried via chat, first time, the other person left the chat and did not come back, and second time i was asked on monday to write to support@, and still no answer.

Is this normal, what kind of delays should i expect ?

Have a great day,
Samuel

1 Like

@sammmmmmmmm Sorry to hear that this has been your experience! I have spoken directly with the support team member you were working with via the ticket. They have reached out to our account team which should be in contact with you soon to work out these questions you have. Thank you for your continued patience.

1 Like

I have the same problem. @sammmmmmmmm, were you able to find an answer? A good way to learn what metric data is ingested into New Relic? A way to drop parts of it that you might not need, in order to reduce costs?

1 Like

Hey @dschridde

I indeed got in touch with someone from the Accounts team as @Joi requested, we were able to exchange a few emails quite rapidly to solve a few other questions, but when it came to my main question “What is all that data, and how do I get a detailed view?” I got that vague answer. Then Silence … Nothing since October 20 … I am very disappointed.

Last answer from the Accounts team:

I’m trying to see if we have a view that can further break down that “metrics” category but if you want a basic understanding of how “metrics” are derived you can use these documents on data types and metric types.

1 Like

@sammmmmmmmm Thanks for sharing about your experience although overall it sounds like it was disappointing. I know our account team does their best to provide the information that would be helpful for you. If the further break down is not a feature that is available then we can put in a request to our product team for them to consider. I’ll followup on this to see if that is the case or not.

2 Likes

@JoiConverse What was the outcome of you following up?

By creating a dashboard and adding a “chart” widget, selecting “data type: metrics” and using the auto-completion for “view a chart with: …” I was able able to figure out that metrics for a lot of Kubernetes cluster internals, not directly related to my application, are ingested into New Relic. By enabling and disabling different agents I was able to deduce which one is responsible.

This is really a cumbersome way to get that information. I would prefer if the New Relic UI could tell me which sources reported a certain metric recently.


New Relic edit

  • I want this, too
  • I have more info to share (reply below)
  • I have a solution for this

0 voters

We take feature ideas seriously and our product managers review every one when plotting their roadmaps. However, there is no guarantee this feature will be implemented. This post ensures the idea is put on the table and discussed though. So please vote and share your extra details with our team.

Hi @dschridde, thank you for the feedback. I will try and follow up with my colleague to get an update but I have gone ahead and submitted a feature idea for your suggestion for an improved New Relic UI. While there is no guarantee that this will be implemented, if it is, we will update the community accordingly.

I’m having the same issue, trying to understand if there is a way to filter out some of the metrics that comes from the agent.

Hi @infrastructure-monit, I’d suggest giving this documentation for data management as it might help with this issue.