Kafka integrations not starting

When we start the infra agent to use the new kafka intragations file we get this error:

time="2019-05-17T10:58:33-04:00" level=debug msg="found plugin instance config file" plugin-instance-path=/etc/newrelic-infra/integrations.d/kafka-config.yml

time=“2019-05-17T10:58:33-04:00” level=debug msg=“no instances loaded for plugin com.newrelic.kafka because it isn’t registered”

time="2019-05-17T10:58:33-04:00" level=debug msg="no plugin configuration found. Using default configuration for all available plugins…"

Hi @j.needham,

The “no instances loaded for plugin com.newrelic.kafka because it isn’t registered” message can arise if your kafka-config has a different integration_name value than the name value in the kafka-definition.yml, per the Infrastructure Integration SDK’s file specification.

Could you double-check those values? If they’re the same, would you mind pasting your config (with any secrets scrubbed) and definition files here?

Hi @j.needham, check if nri-kafka has been installed.

In my case, it was absent.

Thanks for the suggestion @wsantos2!

@j.needham Have you had any luck getting the Kafka integration setup?

Hey,
In my case the integration_name in kafka-config.yml and name in kafka-definition.yml is the same. Still I face the issue. Do you have any more pointers?

Hi @amit.saxena,

Do you have a snippet from verbose logs showing the agent trying to read those files as above? Another issue can arise if the YAML syntax in your config is malformed, which may cause the agent to skip the file and not load the integration. A useful to have is yamllint (typically available in most Linux distribution’s package repositories, and Homebrew) to validate that there are no errors in your file.

time=“2020-03-18T06:42:21Z” level=debug msg=“folder does not exist. Ignoring” component=integrations.DefinitionsRepo folder=/var/db/newrelic-infra/bundled-plugins
time=“2020-03-18T06:42:21Z” level=debug msg=“folder does not exist. Ignoring” component=integrations.DefinitionsRepo folder=/var/db/newrelic-infra/plugins
time=“2020-03-18T06:42:21Z” level=debug msg=“loading config” component=integrations.Manager file=kafka-config.yml folder=/etc/newrelic-infra/integrations.d/
time=“2020-03-18T06:42:21Z” level=debug msg=“skipping v3 integration. To be loaded in another moment” component=integrations.Manager file=kafka-config.yml folder=/etc/newrelic-infra/integrations.d/
time=“2020-03-18T06:42:21Z” level=debug msg=“loading config” component=integrations.Manager file=newrelic-infra.yml folder=/etc/newrelic-infra/integrations.d/
time=“2020-03-18T06:42:21Z” level=warning msg=“can’t load integrations file” component=integrations.Manager error=“yaml: unmarshal errors:\n line 9: cannot unmarshal !!str com.new... into []integration.ConfigEntry” file=newrelic-infra.yml folder=/etc/newrelic-infra/integrations.d/
time=“2020-03-18T06:42:21Z” level=debug msg=“watching /etc/newrelic-infra/integrations.d/” component=integrations.Manager folder=/etc/newrelic-infra/integrations.d/
time=“2020-03-18T06:42:21Z” level=debug msg=“loading integrations from folder” component=integrations.Manager folder=/etc/newrelic-infra/integrations.d/ found=none
time=“2020-03-18T06:42:21Z” level=debug msg=“loading config” component=integrations.Manager file=kafka-config.yml folder=/etc/newrelic-infra/integrations.d
time=“2020-03-18T06:42:21Z” level=debug msg=“skipping v3 integration. To be loaded in another moment” component=integrations.Manager file=kafka-config.yml folder=/etc/newrelic-infra/integrations.d
time=“2020-03-18T06:42:21Z” level=debug msg=“loading config” component=integrations.Manager file=newrelic-infra.yml folder=/etc/newrelic-infra/integrations.d
time=“2020-03-18T06:42:21Z” level=warning msg=“can’t load integrations file” component=integrations.Manager error=“yaml: unmarshal errors:\n line 9: cannot unmarshal !!str com.new... into []integration.ConfigEntry” file=newrelic-infra.yml folder=/etc/newrelic-infra/integrations.d
time=“2020-03-18T06:42:21Z” level=debug msg=“watching /etc/newrelic-infra/integrations.d” component=integrations.Manager folder=/etc/newrelic-infra/integrations.d
time=“2020-03-18T06:42:21Z” level=debug msg=“loading integrations from folder” component=integrations.Manager folder=/etc/newrelic-infra/integrations.d found=none
time=“2020-03-18T06:42:21Z” level=debug msg=“watching /var/db/newrelic-infra/integrations.d” component=integrations.Manager folder=/var/db/newrelic-infra/integrations.d
time=“2020-03-18T06:42:21Z” level=debug msg=“loading integrations from folder” component=integrations.Manager folder=/var/db/newrelic-infra/integrations.d found=none
time=“2020-03-18T06:42:21Z” level=debug msg=“Tried to load an integration where no plugin.yaml was present” component=PluginRegistry path=/var/db/newrelic-infra/newrelic-integrations/bin
time=“2020-03-18T06:42:21Z” level=debug msg=“Found integration definition file” component=PluginRegistry definitionFile=/var/db/newrelic-infra/newrelic-integrations/kafka-definition.yml
time=“2020-03-18T06:42:21Z” level=debug msg=“Integration definition loaded” component=PluginRegistry integration=com.newrelic.kafka
time=“2020-03-18T06:42:21Z” level=debug msg=“Found integration config file” component=PluginRegistry configFile=/etc/newrelic-infra/integrations.d/kafka-config.yml
time=“2020-03-18T06:42:21Z” level=debug msg=“Ignoring directory or non yaml integration config file” component=PluginRegistry path=/etc/newrelic-infra/integrations.d/kafka-config.yml.sample
time=“2020-03-18T06:42:21Z” level=debug msg=“Found integration config file” component=PluginRegistry configFile=/etc/newrelic-infra/integrations.d/newrelic-infra.yml
time=“2020-03-18T06:42:21Z” level=debug msg=“Ignoring v4 integration. To be loaded later.” component=PluginRegistry file=/etc/newrelic-infra/integrations.d/newrelic-infra.yml
time=“2020-03-18T06:42:21Z” level=debug msg=“Found integration config file” component=PluginRegistry configFile=/etc/newrelic-infra/integrations.d/kafka-config.yml
time=“2020-03-18T06:42:21Z” level=debug msg=“Ignoring directory or non yaml integration config file” component=PluginRegistry path=/etc/newrelic-infra/integrations.d/kafka-config.yml.sample
time=“2020-03-18T06:42:21Z” level=debug msg=“Found integration config file” component=PluginRegistry configFile=/etc/newrelic-infra/integrations.d/newrelic-infra.yml
time=“2020-03-18T06:42:21Z” level=debug msg=“Ignoring v4 integration. To be loaded later.” component=PluginRegistry file=/etc/newrelic-infra/integrations.d/newrelic-infra.yml
time=“2020-03-18T06:42:21Z” level=debug msg=“no plugin configuration found. Using default configuration for all available plugins…”

Hello @amit.saxena, I don’t see any errors in the logs. But this line

time=“2020-03-18T06:42:21Z” level=warning msg=“can’t load integrations file” component=integrations.Manager error=“yaml: unmarshal errors:\n line 9: cannot unmarshal !!str `com.new...` into []integration.ConfigEntry” file=newrelic-infra.yml folder=/etc/newrelic-infra/integrations.d/

indicates an issue in the config file. Would you please check? The integration field doesn’t need to be present in newrelic-infra.yml. The integration config file with the integration_name field should be present in the /etc/newrelic-infra/integrations.d/ location.