Can't get logs working... but APM and Infrastructure works fine?

I have Infrastructure and APM monitoring working great. I’d like to monitor logs, too, but I can’t get it working.

When I click on “Logs” in my account, it directs me to install the New Relic Infrastructure Agent and gives me a command to copy and run. Everything goes fine until the logs-integration installation, which hangs at the “Checking for data in New Relic” stage:

==> Installing logs-integration...
  - Which log files would you like to tail? /var/log/messages,/var/log/cloud-init.log,/var/log/secure,/var/log/yum.log
"-": executable file not found in $PATH
"-": executable file not found in $PATH
⠇ Checking for data in New Relic...
❌ Checking for data in New Relic...failed.
==> Installing logs-integration...failed.

ERROR execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/

I’ve read the documentation but it wasn’t helpful. Outgoing port 443 is open on my server, and as mentioned, the APM and Infra monitoring works fine.

What else can I do? This is a CentOS 7 server.

Can you check your logging.d directory for Infrastructure to see if a yml file is being generated with the paths that you specified? If you don’t see a logging.d directory or your yml file in that directory, you may need to manually create one.

1 Like

Thanks for your help, @rstarnes.

Yes, the /etc/newrelic-infra/logging.d/logging.yml file is being generated. I’m just using the defaults.

logs:
  - name: messages
      file: /var/log/messages
  - name: secure
      file: /var/log/secure
  - name: yum.log
      file: /var/log/yum.log

This is the tail end of the installer output:

==> Installing logs-integration...
Which log files would you like to tail? var/log/messages,/var/log/cloud-init.log,/var/log/secure,/var/log/yum.log
"-": executable file not found in $PATH
"-": executable file not found in $PATH
    ❌ Checking for data in New Relic...failed.
==> Installing logs-integration...failed.

ERROR execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/
  One or more integrations failed to install.  Check the install log for more details: /root/.newrelic/newrelic-cli.log
  ---
  Instrumentation recommendations
  We discovered some additional instrumentation opportunities:
  - Java Agent Installer
  - PHP Agent Installer
  - Python Agent Installer
Please refer to the "Data gaps" section in the link to your data.
  ---
  New Relic installation complete!
  Your data is available at https://one.newrelic.com/redirect/entity/[[personal link removed]]
FATAL We encountered an error during the installation: encountered an error while validating receipt of data for logs-integration: reached max validation attempts. If this problem persists please visit the documentation and support page for additional help here: https://one.nr/06vjAeZLKjP

The “guided install” screen in New Relic One shows this:

This is the contents of /root/.newrelic/newrelic-cli.log:

time="2021-03-24T21:58:47Z" level=warning msg="unknown region, using default: US"
time="2021-03-24T21:58:47Z" level=info msg="profile \x1b[36mdefault\x1b[0m added"
time="2021-03-24T21:58:47Z" level=info msg="setting \x1b[36mdefault\x1b[0m as default profile"
time="2021-03-24T22:05:42Z" level=error msg="execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/"
time="2021-03-24T22:05:42Z" level=fatal msg="We encountered an error during the installation: encountered an error while validating receipt of data for logs-integration: reached max validation attempts. If this problem persists please visit the documentation and support page for additional help here: https://one.nr/06vjAeZLKjP"
time="2021-03-24T22:16:36Z" level=error msg="execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/"
time="2021-04-06T00:11:24Z" level=error msg="execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/"
time="2021-04-06T00:22:19Z" level=error msg="execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/"
time="2021-04-06T00:22:19Z" level=fatal msg="We encountered an error during the installation: encountered an error while validating receipt of data for logs-integration: reached max validation attempts. If this problem persists please visit the documentation and support page for additional help here: https://one.nr/06vjAeZLKjP"