New Relic and Kafka cloud - non Java

Good morning,

So my brain is working through making Kafka and making interacting systems more observable, so not only with regards to latency, process time, and maybe some other metrics, but topics available and the messages posted to that topic as well. I really would like to just subscribe NewRelic to a topic as a consumer, have it log that it got it, and then move on.

A lot of this is there for Java ( Java Kafka stuff ), but we are a .NET shop. We are also on the Kafka cloud, so do not have as much control of our installations, so what to do…

I am thinking that I could write a consumer service that subscribes to topics and posts custom events, when new topics come in, via the NR api and that could solve the topic/story part of my problem and I could have that same service ping the Kafka api every x interval for stats, so I guess it could work, but I would really prefer that someone tell me that I am dumb and this has already been done or that there is an easier way, custom Nerdlet, other integration, or whatever.

Has anyone else tried to tackle this? What did you come up with? Thoughts?

Tim

I believe that you are looking for something called New Relic connector for Kafka Connect. The New Relic connector for Kafka Connect allows you to ingest data from Apache Kafka® topics into the New Relic platform without writing a single line of code.

1 Like

It very well could be, thanks for letting me know!

FYI, this didn’t work due to the need to have on-prem connectors, but I still feel it was the right answer. We just have some work to do before we can make use of it. Thanks again for the info.

1 Like