Our first impression (new to NewRelic)

I will outline my thoughts/opinions as I go… hopefully this doesn’t get too long…

First things first, I think this is a big step in the right direction. After my devs asked for these plugins, I started looking at the hodge podge of installation steps spewed into various reviews and was starting to get concerned. We could still never put this on our production servers (#2 and #3 below), but its certainly a lot better than reading through the feedback/reviews to figure out the various steps to install and taking some example init script and modifying it as necessary, etc. Thanks guys!

Now for the feedback…

  1. I think it would make sense if it could do some basic detection of architecture, distro, etc rather than having like 6 scripts to choose from. uname -i and a little checking for certain “release” files would probably go quite a ways to simplify things. I am not sure if its a good example, but I like how http://rvm.io has just one script, I also think its simpler syntax to curl ... | bash rather than use the bash -c "$(curl ..)" syntax, but that may just be my opinion :slight_smile:

  2. Proxy settings? In a PCI environment, direct outbound access is not generally allowed, so a place to set proxy settings would be handy. It would be even handier if it could lookup settings from nrsysmond, but I don’t think that should be required :-/

  3. Native Packages, please! RPM/DEB packages are always easier to support, a yum repository and package installs can be tracked/managed in a RHEL (production) environment. You can then simplify what your script is trying to do.
    … in the Redhat/CentOS case, EPEL has nodejs 0.10.26 already (surprising actually) :smile: … now they may not have all the node_modules you require, but several are there… and you could provide missing ones as proper packages until such time as EPEL included them, or even just embed them into your application installation similar to how the current npi is laid out. Proper packages would then require proper filesystem layout like /etc/newrelic/plugins/(pluginname)/ or /usr/libexec/newrelic-plugins/(plugin-name) … /var/lib/newrelic-plugins maybe? just some thoughts

  4. Also I second what @nikolay says, add a PREFIX option so that it could be installed in /usr/local or whatever. I don’t want my stuff in /home/username/npi…, nor in /root/npi… I’d like to have the plugins run as an unprivileged user (such as newrelic). I think that may be a setting that can be set, but since it installed into /root, that sortof shoots that idea.

  5. I also agree with: Add a non-interactive installation option so that it just “goes” which would make it very useful for automation/kickstart time


… maybe specific to the nrmysql module…

  1. The default config file should at least have the default hostname (hostname -s) instead of “Localhost” (assuming that is the name we will see in NewRelic.

  2. Is it possible to just have it read the config from ~/.my.cnf which we have configured for other reasons?

  3. Ensure system requirements are met, like java is available for plugins that use java? Perhaps offer to install the native JRE package (java-1.7.0-openjdk in RHEL/CentOS).

1 Like

First and foremost, we LOVE long responses containing feedback, so don’t hesitate to make it rain opinions on us in the future! This was certainly very informative and helpful for us so we really appreciate your taking the time!

Now that that’s out of the way, here are my responses to your feedback:

  1. We went down this path a little ways before we ran away with our tails tucked. It is apparently very difficult to reliably detect Linux distros and flavors because there is very little standard in how you do it. Different flavors rely on invoking different executables in different locations and more often than not require you to parse a string for random keywords. That said, I think that now that we have hit a lot of our low-hanging fruit this is a perfectly awesome example of the type of polish that makes the experience that much better. I plan to rewrite the installation script soon, and will bear this in mind!

  2. This one is easy! The NPI tool supports proxy settings:

    ./npi config set proxy_host
    ./npi config set proxy_port
    ./npi config set proxy_username
    ./npi config set proxy_password

Setting them for the NPI tool will also copy them to the plugin’s newrelic.json file as well. All of our testing with this has been fairly contrived, so we would really appreciate your letting us know if this works for your scenarios. As far as integration with nrsysmond…stay tuned, it’s on our radar! :smile:

  1. I anticipate native packages coming, but we wanted to focus on getting this in people’s hands as soon as possible first. That is also the reason that the whole tool is in a single folder (as opposed to the more standard split directory structure). We consciously wanted to make this easy to remove while we iterate so we don’t risk leaving rogue files on your systems while we iterate on the experience. As the tool evolves expect most of these ‘quirks’ to go away though! In regards to Node, we chose to package the distributable to control the environments we have to support (i.e. we get a much smaller test matrix for our team to maintain by controlling what versions of Node we are compatible with).

  2. This one goes back to the last comment, we wanted to make things easy to try out and remove as we iterate on the code–expect that to change as we solidify our experience! As far as directory configuration, this is something I will work on immediately and intend to release by the end of the week. For the user, it defaults to the user running the script but you can change that at any point by running:

    ./npi config set user

As I revisit the install script I’ll look to make both of these options more apparently configurable!

  1. I asked this to Nikolay and I’d be happy to get a response from you as well, do you have examples of tools that you think do this well? Or do you have specific things that you like seeing in these types of tools? Appreciate the recommendations!

Bonus:

  1. Great feedback, at this point there is no place for that code to run for the MySQL plugin specifically, but maybe there is an opportunity to build an extensible solution to help with all plugin configurations! We will look into it!

  2. This gets difficult, we have to balance the convenience for installation vs. the ease of development for our third-party plugin developers (which I strongly recommend you consider becoming!). Since this project was basically engendered to address the concern of configuration, it’s probably not likely we’ll add too many special-cases in the near future. That said, I will bear this feedback in mind while working on vNext items!

  3. Absolutely a great idea to check for the requirements! We’ll consider this as we roll out the next revision. As far as installing the dependency ourselves, we actually received very strong feedback from a lot of ops people that they lock their systems down pretty tightly and do not look kindly upon tools that try to bring in heavy runtimes on their own accord. This problem also compounds itself when we extend scope to other languages with more complex dependencies (e.g. installing C libraries or executable tools like Bundler for Ruby)

Whew, I hope I didn’t put you to sleep, but I wanted to ensure you have an idea of what our thought-process is so you can chime in or correct any misconceptions! Thank you again for taking the time to help us improve our product and please keep the feedback coming!

2 Likes

I find it pretty awesome that you not only read the feedback but responded meaningfully. That is pretty refreshing these days.

  1. I have seen a couple scripts that look like they check for the presence of certain release files in /etc, but that seems like it would be somewhat painful to maintain. A really good idea for that might be to look at how “facter” does it to detect osfamily and operatingsystem. While certainly not a standard, it looks like it wouldn’t be too hard to adapt to a shell script function? (architecture is easy, OS is a bit more involved) For utilities themselves, perhaps gratuitous use of “env” and “which” ? with the ability to override as necessary with environment variables. The installer is becoming its own software project :slight_smile:

  2. RE: Proxy - Awesome! (actually double awesome that it carries the settings into the modules) That is the kind of answer I had hoped for. We will test it out. So maybe this is just documentation enhancement (or my lack of research)… something like ./npi help config or ./npi config help ?? I personally think it becomes cumbersome to document things in multiple places, so I would even be with a link to a wiki or something that outlined the various configuration parameters somehow.

  3. RE: Packages - I figured as much, I am just chiming that in everywhere I go. As I said at the beginning, this is a big step in the right direction. I think it is the absolutely right thing to do while developing. It makes it feel a lot like Apple’s Blah.app folders, and is probably very smart in the end. Perhaps (like we plan to do), we just package up the results of the npi installs. I think that is basically what RHEL/CentOS/EPEL do for stuff like pecl/pear, which feel quite similar to what you are trying to build here. For our initial attempt at packaging, we will probably leave directory structure alone, and just do something like /opt/newrelic-modules or /usr/local/newrelic-modules.

  4. Yep, perhaps it would make sense to look at how “RVM” does that, where they offer a notion of a “shared” or “system-wide” installation in /usr/local, or alternatively install directly to the homedir of the installing user.

  5. The best example I can think of off the cuff would be the ipa-client-install script from FreeIPA, where you specify a bunch of command line options and have a [-U | --unattended] option that gets rid of all the “confirmation” prompts. Another example would be like the Puppet Enterprise installer with an “answers” file. I have seen the concept of an answers file used in several commercial software installations as well.

Bonus-1. Maybe each module could have a sort of “templated” config file, like using [HOSTNAME] or %HOSTNAME%? Then you could either provide the ability to generate the configuration file with certain environment variables set which would be replaced in the template or just copy a “sample” configuration (which is likely what is happening currently)…

Bonus-2. Yea, the further I got into the process, I realized that was probably a bad idea. RETRACTED!

Bonus-3. My recommendation would be to check for requirements (such as the runtime) before install (which java?), and for sure before installing the init script and starting the service. The best thing for now would be to emit a message letting the admin know that they need to install a (java) runtime environment for example, it would be extra nice if it gave OS specific examples. I think this is another good opportunity for a link. It should for sure “exit 1” (or some other non 0 exit code) and give an error message, because it sortof hides any error output, and it just isn’t running currently.

2 Likes

Hey Tommy, I apologize for the delayed response, I’ve been focused on adding Windows and .NET support to NPI, but I wanted to reach out and let you know what changes I’ve made in regards to your feedback. I looked into auto-detecting Linux families and found some very interesting tools but in the end I could not find anything that didn’t fail miserably on certain edge cases, and I didn’t want to risk trying to be too smart and inhibiting people from being able to get the tool on their box at all. I have a ticket pending to add this when the schedule is more permissive, but for now we are going to keep the separate scripts for OS family. In regards to more documentation, we do have formal documentation for the tool now in addition to enhanced READMEs and Plugin Central descriptions. Please feel free to peruse the content and let us know if you have any feedback: http://docs.newrelic.com/docs/plugins/installing-an-npi-compatible-plugin.

Also, we do intend to do packages long term, but it is an arduous problem to solve, so we want to maintain focus on enhancing the core product before we invest heavily in the distribution experience, but I welcome any objections. :smile: Next, I know this is probably more manual than you’d like, but you can now prepend PREFIX= variable to the CURL command to install to a location of your choice. Also, you can prepend “UNATTENDED=true” to bypass the prompt for the installation (I apologize it is not an explicit flag but there is a dirty technical reason why that wouldn’t work for us). It is only a subset of what you asked, but hopefully it is a step in the right direction and makes your use of the NPI tool a little easier! Please don’t hesitate to reach out with any addition questions or concerns and always we love hearing your feedback!

Thanks Adam! I put some documentation feedback in another thread. Happy with progress so far. I tried to package it and I know what you mean. If I come up with a usable spec file I will share it.

1 Like

Spec file as promised:

They produces an RPM that installs the NPI into /opt/newrelic-plugins. It is a “noarch” RPM because I remove the “node” binary and replace it with a symlink to the system “/usr/bin/node” from nodejs. As such, it requires CentOS/RHEL 6 and requires EPEL (which nearly everyone has anyow).

It “works for me” but caveat emptor.

Resulting RPMs:

~tommy

Geez, it won’t be long until we have to put you on the payroll. :stuck_out_tongue:

I really appreciate your taking the time; for the time being I will reference this post for anyone who lets me know they are interested in the RPM route. Thanks for helping make ours–and more importantly–our user’s lives a little easier!

Regards,

Adam

Hi Adam,

Yea, we needed the RPMs for deployment to our datacenter systems, so I figured I might as well share them. We should probably split that message out to its own thread (or just create a new one to track it). They are definitely “use at your own risk” and include my proxy patch, which should be able to be removed in the next version.

~tommy

As mentioned in another thread, I’ve updated the 0.1.5-beta release, pending your verification I will push the released version and then you should be able to rebuild your RPMs. After that, I think you’re absolutely right about spinning up a thread dedicated to this topic (I recommend just creating a new forum post here detailing what you did and how others can contribute and help). Thanks Tommy!

OK, it looks like it works. I am not sure what reason someone would have for having “https://proxy-server.internal.domain” … but prior to working in a PCI environment, I hadn’t seen a proxy in over a decade :smile:

It has my vote of approval.

~tommy

1 Like

I built RPMs for 0.1.5 for CentOS 5 (x86_64) and CentOS6 (noarch). They are available for download from my dropbox. They are, of course, use at your own risk. If you want to build your own RPMs, I have the spec file in my github.