Can't get logs working... but APM and Infrastructure works fine?

I have Infrastructure and APM monitoring working great. I’d like to monitor logs, too, but I can’t get it working.

When I click on “Logs” in my account, it directs me to install the New Relic Infrastructure Agent and gives me a command to copy and run. Everything goes fine until the logs-integration installation, which hangs at the “Checking for data in New Relic” stage:

==> Installing logs-integration...
  - Which log files would you like to tail? /var/log/messages,/var/log/cloud-init.log,/var/log/secure,/var/log/yum.log
"-": executable file not found in $PATH
"-": executable file not found in $PATH
⠇ Checking for data in New Relic...
❌ Checking for data in New Relic...failed.
==> Installing logs-integration...failed.

ERROR execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/

I’ve read the documentation but it wasn’t helpful. Outgoing port 443 is open on my server, and as mentioned, the APM and Infra monitoring works fine.

What else can I do? This is a CentOS 7 server.

Can you check your logging.d directory for Infrastructure to see if a yml file is being generated with the paths that you specified? If you don’t see a logging.d directory or your yml file in that directory, you may need to manually create one.

1 Like

Thanks for your help, @rstarnes.

Yes, the /etc/newrelic-infra/logging.d/logging.yml file is being generated. I’m just using the defaults.

logs:
  - name: messages
      file: /var/log/messages
  - name: secure
      file: /var/log/secure
  - name: yum.log
      file: /var/log/yum.log

This is the tail end of the installer output:

==> Installing logs-integration...
Which log files would you like to tail? var/log/messages,/var/log/cloud-init.log,/var/log/secure,/var/log/yum.log
"-": executable file not found in $PATH
"-": executable file not found in $PATH
    ❌ Checking for data in New Relic...failed.
==> Installing logs-integration...failed.

ERROR execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/
  One or more integrations failed to install.  Check the install log for more details: /root/.newrelic/newrelic-cli.log
  ---
  Instrumentation recommendations
  We discovered some additional instrumentation opportunities:
  - Java Agent Installer
  - PHP Agent Installer
  - Python Agent Installer
Please refer to the "Data gaps" section in the link to your data.
  ---
  New Relic installation complete!
  Your data is available at https://one.newrelic.com/redirect/entity/[[personal link removed]]
FATAL We encountered an error during the installation: encountered an error while validating receipt of data for logs-integration: reached max validation attempts. If this problem persists please visit the documentation and support page for additional help here: https://one.nr/06vjAeZLKjP

The “guided install” screen in New Relic One shows this:

This is the contents of /root/.newrelic/newrelic-cli.log:

time="2021-03-24T21:58:47Z" level=warning msg="unknown region, using default: US"
time="2021-03-24T21:58:47Z" level=info msg="profile \x1b[36mdefault\x1b[0m added"
time="2021-03-24T21:58:47Z" level=info msg="setting \x1b[36mdefault\x1b[0m as default profile"
time="2021-03-24T22:05:42Z" level=error msg="execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/"
time="2021-03-24T22:05:42Z" level=fatal msg="We encountered an error during the installation: encountered an error while validating receipt of data for logs-integration: reached max validation attempts. If this problem persists please visit the documentation and support page for additional help here: https://one.nr/06vjAeZLKjP"
time="2021-03-24T22:16:36Z" level=error msg="execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/"
time="2021-04-06T00:11:24Z" level=error msg="execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/"
time="2021-04-06T00:22:19Z" level=error msg="execution of logs-integration failed, please see the following link for clues on how to resolve the issue: https://docs.newrelic.com/docs/using-new-relic/cross-product-functions/troubleshooting/not-seeing-data/"
time="2021-04-06T00:22:19Z" level=fatal msg="We encountered an error during the installation: encountered an error while validating receipt of data for logs-integration: reached max validation attempts. If this problem persists please visit the documentation and support page for additional help here: https://one.nr/06vjAeZLKjP"

@jordangreer Following up to see how this is working for you? Seems like @rstarnes suggestions were helpful.

No. With no further assistance in this topic, I had no choice but to give up on New Relic logging.

@jordangreer Sorry to hear that. I am following up with our support team to see if there is anything further they can suggest.

I am also facing the same issue while installing application logs and log integration just fails

Same problem here while trying to setup log ingestion.

@jordangreer @SathiyarajManoharan @jeremy47

Do or did any of you already have the Infrastructure agent installed when attempting to go through the CLI guided install? Since this issue was originally created our engineers have identified (and I believe they have fixed) an issue with the log installation portion of the guided install when customer’s have existing New Relic setups (IE not first time setup).

The issue should be fixed, but if you still see issues, you can try is checking your ~/.newrelic directory to see if there are any pre-existing account credentials in there that don’t match up with the ones being used for the CLI installation. If they don’t match up then that is likely the issue:

  • Run cat ~/.newrelic/default-profile.json and then check whether or not it correlates to the account ID and/or API key in your credentials file: cat ~/.newrelic/credentials.json.

If the credentials dont match up you can try renaming the .newrelic directory temporarily and then running the CLI script again to see if the installation goes through. As a heads up though the keys contained in those files are currently being used by your existing products, so renaming and reinstalling may have a unintended side effects.

All the above said though, you can get around all of this by just manually setting up logging for Infrastructure rather than deciphering credentials:

I hope that’s helpful for you all.

I did have the Infrastructure agent installed previously, and unfortunately I still encounter the same issue. I did try renaming the .newrelic directory as suggested, but that had no effect.

As the CLI installer begins it confirms it can reach NewRelic’s servers, but when it tries again in the logging setup, it always fails. APM and Infrastructure monitoring continue to work normally.

I have followed those instructions and configured a logging.yml file. However, when I click on “Logs” in the New Relic ONE launcher header, it forces me to the CLI installer instructions again, which of course do not work. I assume if you are not able to successfully setup logging during the CLI installation process, it will not let you past this screen.

1 Like

We are facing the same error. Hopefully there will be a fix soon?

1 Like

Same issue here as well. New installation. It would be welcomed to get a fix for that.

1 Like

Same issue here as well. New install Ubuntu 20 LTS and latest MariaDB.

1 Like

Same issue Ami Linux 2

1 Like

the same issue Red HAT 6 anyone found solution?

1 Like

I’m having the same issue on ubuntu 20.04

1 Like