Unable to get Broker metrics - KafkaBrokerSample - Kafka Integration

This is my config file snippet:

  • name: kafka-metrics-bootstrap-discovery
    command: metrics
    arguments:
    cluster_name: “npcluster1”
    autodiscover_strategy: “bootstrap”
    bootstrap_broker_host: “”
    bootstrap_broker_kafka_port: 9092
    bootstrap_broker_kafka_protocol: SSL
    bootstrap_broker_jmx_port: 9999
    tls_cert_file: /app/kafka/keystore/iaas.crt
    tls_key_file: /app/kafka/keystore/iaas.key
    tls_ca_file: /app/kafka/keystore/DigiCertCA.crt
    tls_insecure_skip_verify: true
    collect_broker_topic_data: true
    topic_mode: ‘all’
    labels:
    env: NONPROD
    role: Kafka-NP-Broker

    • name: kafka-inventory
      command: inventory
      arguments:
      cluster_name: “npcluster1”
      autodiscover_strategy: “bootstrap”
      bootstrap_broker_host: “”
      bootstrap_broker_kafka_port: 9092
      bootstrap_broker_kafka_protocol: SSL
      bootstrap_broker_jmx_port: 9999
      topic_mode: ‘all’
      collect_broker_topic_data: true
      tls_cert_file: /app/kafka/keystore/iaas.crt
      tls_key_file: /app/kafka/keystore/iaas.key
      tls_ca_file: /app/kafka/keystore/DigiCertCA.crt
      tls_insecure_skip_verify: true
      labels:
      env: NONPROD
      role: Kafka-NP-Broker

Basically I am unable to get the Broker Metrics in NR insights. I tested the jmx connectivity with the help of JMX tool and I see the metrics. See below -

bin/kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec
Trying to connect to JMX url: service:jmx:rmi:///jndi/rmi://:9999/jmxrmi.
“time”,“kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:Count”,“kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:EventType”,“kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:FifteenMinuteRate”,“kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:FiveMinuteRate”,“kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:MeanRate”,“kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:OneMinuteRate”,“kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:RateUnit”
1627585122400,661,messages,0.6750345074293932,1.7362635252385221,4.372243862108869,4.035268064882846,SECONDS
1627585124400,672,messages,0.6750345074293932,1.7362635252385221,4.386981997704842,4.035268064882846,SECONDS
1627585126400,677,messages,0.6923472912376871,1.7703740888616295,4.3626393981574365,4.016457069036111,SECONDS

The error I get in nr log file -
Unable to execute JMX query for MBean ‘kafka.network:type=RequestMetrics,name=TotalTim
eMs,request=Metadata’: EOF\n[ERR] Unable to execute JMX query for MBean ‘kafka.network:type=RequestMetrics,name=TotalTimeMs,request=Fetch’: writing nrjmx stdin: write |1: broken pipe\n[ERR] Unable to execute JM
X query for MBean ‘kafka.network:type=RequestMetrics,name=TotalTimeMs,request=Offsets’: writing nrjmx stdin: write |1: broken pipe\n[ERR] Unable to execute JMX query for MBean ‘kafka.network:type=RequestMetrics
,name=TotalTimeMs,request=UpdateMetadata’: writing nrjmx stdin: write |1: broken pipe\n[ERR] Unable to execute JMX query for MBean ‘kafka.network:type=RequestMetrics,name=TotalTimeMs,request=Produce’: writing n
rjmx stdin: write |1: broken pipe\n[ERR] Unable to execute JMX query for MBean ‘kafka.server:type=ReplicaManager,name=’: writing nrjmx stdin: write |1: broken pipe\n[ERR] Unable to execute JMX query for MBean
'kafka.controller:type=ControllerStats,name=
’: writing nrjmx stdin: write |1: broken pipe\n[ERR] Unable to execute JMX query for MBean ‘kafka.server:type=BrokerTopicMetrics,name=’: writing nrjmx stdin: write
|1: broken pipe\n[ERR] Unable to execute JMX query for MBean 'kafka.log:type=LogFlushStats,name=
’: writing nrjmx stdin: write |1: broken pipe\n[ERR] Unable to execute JMX query for MBean ‘kafka.server:type=Kaf
kaRequestHandlerPool,name=RequestHandlerAvgIdlePercent’: writing nrjmx stdin: write |1: broken pipe\n[ERR] Unable to execute JMX query for MBean ‘kafka.server:type=DelayedFetchMetrics,name=ExpiresPerSec,fetcher
Type=’: writing nrjmx stdin: write |1: broken pipe\n[ERR] Unable to execute JMX query for MBean 'kafka.network:type=RequestMetrics,name=RequestsPerSec,request=,*’: writing nrjmx stdin: write |1: broken pipe\n
[ERR] Unable to execute JMX query for MBean ‘kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec,topic=soi_ivr_outbound_call’: writing nrjmx stdin: write |1: broken pipe\n

@mohammed.khan1 Could you provide the version of the infrastructure agent and nri-kafka integration. If the agent version is v1.8+ you should be able to use the v4 configuration format and we suggest that you convert this to that format in order to eliminate the old formatting being the issue. You can find the v4 configuration examples for kafka in the nri-kafka github. This should eliminate the need for having two blocks for inventory and metrics along with simplifying the configuration. If after that change you are still having issues please update this post and we will try to get back to you as soon as possible.

This topic was automatically closed after 365 days. New replies are no longer allowed.