Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Relic Solution: Troubleshooting the Kafka On-Host Integration

kafka
on-host-integrations

#1

The Kafka OnHost Integration is one of the available Infrastructure On-Host integrations available. This article will go into some of the more common issues with it.


The first thing to do is to make sure that Java_Home is accessible by the Infra Agent. You may see an error similiar to this:

Unable to execute JMX query for MBean ‘kafka.consumer:type=consumer-fetch-manager-metrics,client-id=connect-analyticsMongo,topic=*’: JMX tool exited with error: exit status 127

The following can help to make sure Java is installed and that the path is correct:

which java
sudo cat /proc/$(pgrep java)/environ
sudo cat /proc/$(pgrep newrelic-infra)/environ

A way to pass JAVA_HOME to the Infra Agent is to add it as an environmental variable. You would add the following to your newrelic-infra.yml config file:

passthrough_environment:
  - JAVA_HOME

One of the more common items that trips people up is not having JMX enabled. You’ll want to ensure that JMX is enabled on your zookeeper_hosts , and that the JMX port is network accessible to the host running the integration.

Relating to this, the Kafka integration only supports collecting metrics from Java applications with JMX enabled. So any applications that do not have JMX enabled won’t be collected.

A good check is to run the following to verify that JMX is configured and listening:

cat /proc/$(pgrep java)/cmdline
ss -ltnp | grep java

One way to test that data is being collected is to run the following manually through JMX to see if data is coming back. You’ll want to adjust the above to your environment.

echo ‘kafka.producer:type=producer-topic-metrics,client-id=MyKafkaProducer,topic=’ | nrjmx -host localhost -port 9989
echo 'kafka.consumer:type=ZookeeperConsumerConnector,name=
,clientId=consumer-1’ | nrjmx -host localhost -port 9987

Additionally, a useful resource for debugging issues with JMX is this table from Oracle:

https://docs.oracle.com/javase/6/docs/technotes/guides/management/agent.html#gdeum