Auto-scaling kubernetes pods with metric data stored in NewRelic

Support: Full-Stack Observability (FSO) Infrastructure
, , ,
#1

We are in the process of researching the approach of auto-scaling kubernetes pods with metric data stored in NewRelic. Our application code is reporting custom metrics with the NewRelic Java agent, and we would like to make use of it for scaling the pods in Kubernetes. I did go through the NewRelic documentation and user forums about Kubernetes integration, NewRelic Infrastructure Agent, and Prometheus integration. We could use the NewRelic Alert workflow to trigger a pipeline which scales the pods. The other method is using Kubernetes Metric Server to scale pods via Horizontal Pod Scaler. There exist a rudimentary/unofficial integration GitHub - FlexShopper/newrelic-custom-metrics. Other solutions like Prometheus, StackDriver, DataDog have an official supported integration with Kubernetes Metric Server, which allows defining scale conditions for Horizontal Pod AutoScaler using their metrics data.

What is your recommendation to auto-scale pods with NewRelic custom metrics?

6 Likes
Feature idea: Ability to auto-scale k8s pods with NewRelic custom metrics
#2

Hi @mgay,

To my knowledge, we don’t currently have a recommended method for using custom metrics in New Relic for Horizontal Pod Autoscaling in Kubernetes, we’re starting to set out a lot of the groundwork that would make this feasible. This would be a great suggestion to bring over to the Feature Ideas thread.

Feature idea: Ability to auto-scale k8s pods with NewRelic custom metrics
#3

+1 to this. I’ve entered this as a feature idea:

New Relic adapter for scaling app using HPA with custom metrics
#4

Thanks for posting that @dalvizu :slight_smile:

#5

Hi @mgay

I’ve forked the k8s-cloudwatch-adapter into this https://github.com/kuperiu/k8s-newrelic-adapter and now it works with newrelic queries.
You just need to add the query to the external metric and it will wotk

2 Likes
#6

Nice! Thanks for sharing here @kuperiu

#7

Hello @kuperiu,

I will install the adapter in my test environment and apply our NRQL query.

1 Like
#8

Do let us know how it goes for you :smiley:

#9

Nice !

Every news about this ?

#10

Hi @xavier.krantz! In regards to the feature idea mentioned above, it will not be implemented unfortunately. Hopefully the other community members on this thread will have updates in regards to their workarounds.

#11

Hi @nmcnamara

Thanks for the update !

I have missed the message you posted about the FeatureIdea being refused on the mentioned topic :sweat_smile:

As I have just posted on the FeatureIdea topic as a reply, it is a bit of shame to not implement such thing. Datadog does and it would be really useful to leverage all the data we send to you to be able to adapt dynamically rather that make some hack on our side …

I am a bit disappointed to see such lack of Kubernetes integration.
I hope the Operator is going to have more resources soon

#12

I appreciate where you’re coming from but hopefully this will be considered at a future date!

#13

Hi all, in response to requests like yours, we have just launched a beta version of an official New Relic custom metrics adapter - please refer to this post for more details and feel free to share any feedback via the repo!

1 Like