Hi. I’d like to know if there’s any way to reduce the amount of ingested data coming from kuberentes integration by increasing the sampling interval or such. We’re using the nri-bundle for kubernetes, and you can tune the sampling interval for hosts metrics, however there’s no configuration available for all the kubernetes metrics that kube-state-metrics exposes. Is there a way to do so?
Thank you.
Hi @ferran3, welcome to the Explorers Hub.
One option would be to disable ksm metrics:
This would have the following effect:
Disabling kube-state-metrics also disables data collection for the following:
ReplicaSets
DaemonSets
StatefulSets
Namespaces
Deployments
Services
Endpoints
Pods (that are pending)
Additionally, disabling this affects the Kubernetes Cluster Explorer in the following ways:
No pending pods are shown.
No filters based on services.
You can also limit which processes to send to New relic as well as changing the process sample rate.
Could you be more specific and show us how to do this with helm? Disabling the ksm is pretty trivial with helm, but modifying the yaml file for the infrastructure agent doesn’t, exactly, work the same way the documentation provides…
For example, If I use the following command to deploy the nri-bundle, can you confirm that it will, indeed, change the storage_sample_rate to once every 50 seconds?
helm upgrade
–install newrelic-bundle newrelic/nri-bundle
–namespace=default
–set global.licenseKey=MyKey
–set global.cluster=MyCluster
–set kubeEvents.enabled=true
–set webhook.enabled=true
–set prometheus.enabled=true
–set logging.enabled=true
–set ksm.enabled=true
–set infrastructure.metrics_storage_sample_rate=50
I’m still kind of New Relic newb, so I don’t know how to check if it’s working or not… my clusters vary pretty wildly so it’s hard to just look at something like the changes in storage sampling when you’re constantly scaling.
The documentation on helm, certainly leaves a lot to be desired.