Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Ubuntu: Starting New Relic Server Monitor: newrelic-sysmond FAILED

servermonitor
ubuntu

#1

Please paste the permalink to the page in question below:

https://docs.newrelic.com/docs/servers/new-relic-servers-linux/installation-configuration/servers-installation-ubuntu-debian

Please share your agent and other relevant versions below:

$ nrsysmond -h
New Relic system monitor daemon version 2.3.0.132 ("cloudy") ("8cff29599e1e9619b31da739b2ac86c7213f6b71")
(C) Copyright 2009-2016 New Relic Inc. All rights reserved.
curl version libcurl/7.42.1 OpenSSL/1.0.1t zlib/1.2.8
pcre version 8.36 2014-09-26
zlib version 1.2.8

Please share your question/describe your issue below. Include any screenshots that may help us understand your question:

I followed the instructions for installing the server monitoring client on Ubuntu, but the daemon will not start. The configuration at /etc/newrelic/nrsysmond.cfg contains the license key as expected. The output from attempting to start the client looks like this:

$ sudo /etc/init.d/newrelic-sysmond start
Starting New Relic Server Monitor: newrelic-sysmond FAILED

The log file at /var/log/newrelic/nrsysmond.log is empty:

$ ls -alh /var/log/newrelic/nrsysmond.log
-rw-r--r-- 1 newrelic newrelic 0 Aug  3 14:56 /var/log/newrelic/nrsysmond.log

Other things I’ve tried:

  • Adding the newrelic user to the newrelic group (though I think it was already in there):
$ usermod -a -G newrelic newrelic
  • Resetting the permissions on the log folder:
$ sudo chown newrelic:newrelic /var/log/newrelic
  • Resetting the license key:
$ sudo nrsysmond-config --set license_key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Any ideas on how I can get this working?


#2

Hi @danielsamuels,

Could you let us know what version of Ubuntu you are using? If you are unsure you can find out using the following command:

lsb_release -a

Additionally could you please check the local permissions on the New Relic directory?

ls -la /etc/newrelic/


#3
$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:   	Ubuntu 14.04.3 LTS
Release:       	14.04
Codename:      	trusty
$ ls -la /etc/newrelic/
total 32
drwxr-xr-x  2 root     root     4096 Aug  3 14:56 .
drwxr-xr-x 98 root     root     4096 Aug 25 06:38 ..
-rw-------  1 newrelic newrelic 9735 Aug 23 11:43 nrsysmond.cfg
-rw-------  1 root     root     9735 Aug 23 11:43 nrsysmond.cfg.bak

#4

@danielsamuels as quick test, and hopefully a way to get the startup failure message to report, could you start the server agent manually with the following command and paste the output?

su newrelic -s /bin/sh -c "env -i /usr/sbin/nrsysmond -f -c /etc/newrelic/nrsysmond.cfg -p /var/run/newrelic/nrsysmond.pid"

It’s expected to see nothing at the logs since the server agent fails at startup before it has a chance to write anything to the log file. The above command should reveal the underlying problem through STDOUT though.


#5
$ sudo su newrelic -s /bin/sh -c "env -i /usr/sbin/nrsysmond -f -c /etc/newrelic/nrsysmond.cfg -p /var/run/newrelic/nrsysmond.pid"
ERROR: log file '/var/log/newrelic/nrsysmond.log' could not be opened - cannot continue.

#6

Aw, yeah! That looks to be the issue then.

I would ensure the file at /var/log/newrelic/nrsysmond.log exists and is read/writeable by the newrelic user and group (which should have been created during the installation process).

chmod 644 /var/log/newrelic/nrsysmond.log and chown newrelic:newrelic /var/log/newrelic/nrsysmond.log should do the trick.

It’s unclear why the file wasn’t already there, but I’d be curious to see if the standard startup of /etc/init.d/newrelic-sysmond start still fails after ensuring the log file exists and has the proper permissions.


#7

Unfortunately that didn’t fix things.

$ ls -al /var/log/newrelic/
total 8
drwxr-xr-x  2 newrelic   newrelic 4096 Aug  3 14:57 .
drwxrwxr-- 13 clientname webapps  4096 Sep 11 06:29 ..
-rw-r--r--  1 newrelic   newrelic    0 Aug  3 14:56 nrsysmond.log
$ sudo su newrelic -s /bin/sh -c "env -i /usr/sbin/nrsysmond -f -c /etc/newrelic/nrsysmond.cfg -p /var/run/newrelic/nrsysmond.pid"
ERROR: log file '/var/log/newrelic/nrsysmond.log' could not be opened - cannot continue.

#8

@danielsamuels

You have succeeded in changing the permissions on the log file for the newrelic user here, however the nrsysmond.log file still cannot be opened. This could be residual permissions issue, or perhaps a flaw of how the sudo password is entered in the terminal.

Could you try running a simple command as sudo before running the command @dulan provided? This will ensure your password is entered, ensuring sudo permissions to the currently running terminal window.

For example first run sudo ls and enter your password. Then run:

sudo su newrelic -s /bin/sh -c "env -i /usr/sbin/nrsysmond -f -c /etc/newrelic/nrsysmond.cfg -p /var/run/newrelic/nrsysmond.pid"

If you are still having problems, it could be because some security settings associate file permissions to the process that created them. For example the current log file /var/log/newrelic/nrsysmond.log could belong to an older sysmond process, preventing the new sysmond process from writing to it. In that case if you simply delete the file at /var/log/newrelic/nrsysmond.log it should allow the new process to create a new log file to which it has permissions to write.

Please let us know how it goes with those two suggestions!


#9

Unfortunately neither of those worked either. Sudoing from this user doesn’t require a password, so it’s definitely not an input problem. Even going into an interactive root shell (sudo -i) and running the commands was giving the same result.

I’ve updated the config to point the log file to /tmp/nrsysmond.log, which works. But obviously this isn’t ideal.


#10

@danielsamuels from your post earlier (with permissions for the directory), I wonder if it might still be that the agent doesn’t have access all the way down into the log file at that location.

In particular, I noticed that the parent dir (/var/log) isn’t world-executable, which might be preventing the newrelic user from opening the directory:

drwxrwxr-- 13 clientname webapps  4096 Sep 11 06:29 ..

The user/group/permissions I’m seeing there are a bit non-standard for most Linux distributions, but ultimately the newrelic just needs to have read & entry access to each directory along the whole path /var/log/newrelic in addition to read & write access on the log file itself.

If changing permissions to that extent isn’t feasible for other reasons, then the alternate log file is the next best option.


#11

I had the same error as you. For me it resolved deleting the pid file /var/run/newrelic/nrsysmond.pid

Probably the startup procedure checks if the pid file exists, in which case it does not want to start.


September 23, 2017—Post of the Week: Synthetics, Alerts, NRQL and more!
#12