October 2018 Coffee Chat: Best Practices & Getting started with New Relic

October Coffee Chat :coffee:

October, as you likely noticed, is all about validating your New Relic knowledge. We hope you participate with us on October 24th for some real-time chit chat with the experts of best practices, our Solutions Architects!

What can you expect from a Coffee Chat ? It’s an hour dedicated to learning from New Relic experts! We move a conversation topic to slack and you can pepper an expert with questions in real time. I then take that conversation and publish it here, in the community! (Check out this one, and this one !)

Coffee Chats are a great opportunity to learn some proven practices, share knowledge with each other in a casual setting, and share a virtual cup of coffee or tea with other New Relic Explorers.

Coffee Chat with the Solutions Architect Team

This team are the experts when customers need to get up to speed—they know New Relic best practices like the back of their hand! See details below.

How to join the Coffee Chat Slack channel

To get invited to our New Relic Users Slack workspace follow the instructions below:

  1. Head over to the sign up page: https://newrelicusers-signup.herokuapp.com/
  2. Enter in your email address. Pretty soon you’ll receive an invite in your email inbox from Slack to join the New Relic Users workspace.
  3. Once you’re in the correct workspace, either through your browser or the Slack app you can navigate to the #coffee-chat channel . You’re welcome to also come to #introduce-yourself to tell us a bit about you, and what your New Relic specialty subjects are :slight_smile:
  4. At the time the Coffee Chat is due to begin, come on into the #coffee-chat channel to ask your questions.

Note that this workspace isn’t monitored outside of scheduled Coffee Chats. If you do have questions outside of the timeframe of the scheduled coffee chats, please post that question on the Explorers Hub.

See you on the 24th! :blush:



Coffee Chat with The Associate Solutions Architects: @jwilcox @sschneider @twittinger !

October 24, 2018 conversation from #coffee-chat slack channel, New Relic Users.

@Linds [9:47 AM]
Before we officially get started, please do a :wave: emoji reaction if you have taken and/or PASSED any of our Best Practice Quizzes this month!

@Linds [10:00 AM]
Welcome to our Coffee Chat! We have @Tony @Steve @Jimmy here with us today - these guys are New Relic experts: Associate Solutions Architects…

What does that even mean?! How about an intro?

@twittinger [10:02 AM]
Hi everyone I’ve been at New Relic for 2+ years and I help enable customers to be successful using the New Relic platform.

@jwilcox [10:02 AM]
TONY. Spencer looks so good in your picture. What a large cat.

(added for cat reference)

@sschneider [10:02 AM]
:wave: Hi! I’m Steve, been with New Relic for a few years in Support and now this role. I spend a lot of time on the forums so you might have seen my already :wink: ASA means “customer enablement”, we help you get up and running with New Relic using best practices, so you’re ready to hit the ground running in monitoring your data

Also that cat. Wow.

@twittinger [10:03 AM]
He falls under the less active category.

@sschneider [10:03 AM]
Also plug for my corgi Spock - here he is being lazy as can be:

@jwilcox [10:04 AM]
Jimmy here :smile: I’ve been here 2 years and also came from Support. I’m a people person and nothing delights me more than helping customers solve their problems.

@Linds [10:04 AM]
FYI All of you should feel free to post pet pictures at any time during this coffee chat

@sschneider [10:05 AM]
For every answer the ASA’s provide, we will take one cute cat/dog picture as payment :stuck_out_tongue:

@Linds [10:05 AM]
ANYWAY. New Relic Best Practices, where should customers start?!

@Louis [10:05 AM]
joined #coffee-chat.

@jwilcox [10:08 AM]
When getting set up with New Relic, the first thing to consider is how you will name your apps in New Relic. When monitoring multiple services, it’s good to consider a naming schema that will allow your team(s) to quickly navigate to the apps they want to check out.

Here’s a great blog post that goes into more detail: https://blog.newrelic.com/engineering/app-naming-conventions/

@JohnGillotti [10:08 AM]

@anon85944545 [10:13 AM]
As an ASA, how do you advise customers to manage enterprise-scale architectural designs for NR? our org has about 100 subAccounts, and we manage the entire platform with 3 FTEs; so we’ve automated and scripted as much as possible, but we hit snags when trying to script the actual account management portions (where there is no API support currently).

For example, to run a script to onboard a new team’s subAccount with our default alerts; we have to manually build the subAccount, then manually generate API keys for the Account/Admin/NRQL portions; we’d :heart: to remove all the instances where we have to use the word manual

@sschneider [10:15 AM]
Ooh boy! We are right there with you about that too. Right now there isn’t much in the way of automation options beyond what you’ve already found. But, we hear a lot about this ask and our product teams have developed their roadmaps for this accordingly

I can’t say too much more beyond that officially, but know that your voice is being heard (and you’re not alone!)

We’re gonna check if one exists, but have you added a feature idea in the forum for this yet @anon85944545?

@jwilcox [10:17 AM]
You’re definitely not alone :grin: When I was in Support, we got a lot of tickets about account related APIs

@anon85944545 [10:18 AM]
I am not positive tbh, we worked on the issue with our TAM and he provided some interesting ideas re: Selenium scripting and browser extensions. Then our org put security blocks on browser extensions a week later :cry:

@sschneider [10:18 AM]
Right, that would have been my guess too. Security teams aren’t a big fan of running scripts like that at all :sleuth_or_spy:

@anon85944545 [10:19 AM]
It’s understandable, but ultimately puts the onus back on vendors to provide more robust API capabilities (which you’re in the process of). I just wanted to make sure we’re not unknowingly putting ourselves through heartache and there wasn’t some magical trick that we were obtuse to.

@sschneider [10:20 AM]
So, one other painful option would be to use the Synthetics API and a scripted browser to automate that process

Yeah totally, we’ll happily announce that stuff as soon as it comes out :smile:

@anon85944545 [10:21 AM]
yeah, I thought about that but the time it would take to build a scalable solution in Synthetics would be better spent on other automation items in our team tbh.
If it’s not RESTful, it’s painful.

@sschneider [10:21 AM]
Most definitely :smile:

@MonitoringLife [10:21 AM]
@anon85944545 is there anything in the partner API that would help you automate the sub account stuff?

@sschneider [10:21 AM]
Possible? yes. Painful? Oh yes.

@anon85944545 [10:22 AM]
We asked for access to the Partner API, our TAM investigated and found that some of the functions we needed are not available in there either. I want to say that you cannot create accounts, or API keys with that API, only GET items.

@sschneider [10:23 AM]
Hmm you should be able to use the create option: https://docs.newrelic.com/docs/new-relic-partnerships/partnerships/partner-api/partnership-api-account-object#account-calls

@anon85944545 [10:24 AM]
I mean, if someone wants to give us access to the Partner API we’ll definitely be test dummies

@sschneider [10:24 AM]
But there could be more going on behind the scenes that makes it not a viable option for you

@anon85944545 [10:24 AM]

@sschneider [10:24 AM]

This coffee chat got infinitely less productive knowing that giphy works in here too

@anon85944545 [10:25 AM]
most definitely; I am 100% on board with creating ~chaos~ solutions; we’re willing to try, but as we’re not partners, we don’t have access right now

@twittinger [10:29 AM]
There were some forum requests about Alerts, what are some of your alert challenges?

@Linds [10:30 AM]
@here @here @here @jbiggley ^^ :wink: :alarm_clock:

@anon85944545 [10:30 AM]
:joy: @Linds - he’s currently training our run team on new relic; not sure he can join today

@hross [10:32 AM]
Reverse question for our customer friends - what’s really working well for you with Alerts. What did you get RIGHT?

As an option

@MonitoringLife [10:34 AM]
We had a little trouble finding the alerts related to the circuit breaker going off, just wanted to share it ‘AgentCheck/CircuitBreaker/tripped/all’ under ‘search metric names’

@anon85944545 [10:35 AM]
We’ve successfully onboarded (as much as we can, there’s more we WANT to do) all of our infrastructure alerts via script and the alert API by creating NRQL alerts; this is a work-around to using the Infra API as alerts generated there can only be further edited via the API (not in console). It’s not ideal as there’s still NRQL tech debt, but it’s functional and allowed us to meet goals.

No lie, @Linds, this is from Josh, note the time stamp is AFTER my post above. (it’s scary how much we sync on things) #BrotherFromAnotherMother

@MonitoringLife [10:38 AM]
Would be cool to see some greater flexibility in the Synthetics alerts. Number of retries, time space between the checks, etc.

@JohnGillotti [10:39 AM]
@MonitoringLife there is an enhancement request for synthetics that has been back burnered - don’t know why though

@twittinger [10:40 AM]
@anon85944545 The Infra API / Alerts API thing is def a challenge. this is another area that is on the roadmap
When you refer to NRQL tech debt is that learning NRQL or something else?

@MonitoringLife [10:44 AM]
are you using NRQL do avoid having to specify entities?

@anon85944545 [10:44 AM]
One of the pain points we have is that some integer values are stored in Insights as string; which makes it impossible to evaluate against in alerts. The one on top of my head is CPU Core Count; we use that as a correlation to CPU utilization in our on-prem solution; right now in NR we have the following:

SELECT MAX(loadAverageOneMinute) FROM SystemSample FACET entityName WHERE coreCount = 'YY'
where YYis a static value of 1|2|4|8|16 and the critical threshold is 3xYY (3|6|12|24|48) sustained for a period of time.

This results in 5 distinct alert conditions. If the value was stored as an integer, we could do a single condition with a comparison operator. It’s small, but definitely a low hanging fruit scenario for improvements.

@twittinger - the tech debt is in our app teams needing to learn a new query language, yes.

@sschneider [10:47 AM]
Try giving this query a shot instead: SELECT MAX(numeric(loadAverageOneMinute)) FROM SystemSample FACET entityName WHERE coreCount = 'YY'

@anon85944545 [10:48 AM]
To be fair; NRQL is a fairly intuitive language; it’d be REALLY great to see a series of video tutorials that go over the different functions and whatnot though. not everyone knows SQL as a starting point like we did, so I’m not sure our experience in ease would be similar to someone coming from a different background
@Steve - what we want to do is not have to statically set the WHERE clause; we’d like to be able to come up with a solution that aligns with WHERE MAX(numeric(loadAverageOneMinute)) * 3 > coreCount; then the critical threshold would be 1

The hurdle we (@jbiggley really) hit was that coreCount is stored as a string
@MonitoringLife - (if that question was directed to me) how do you mean?

@sschneider [10:52 AM]
Gotcha. Numeric() will at least help with coreCount being stored as a string

@anon85944545 [10:52 AM]
can we throw something like numeric( coreCount ) ?
i’ll test that today
i had thought there was not a function to convert data types; but it’s been a few months

@Linds [10:54 AM]
Let us know how that test goes @anon85944545 in the community!

@jwilcox [10:57 AM]
@anon85944545 you get a new feature! and @MonitoringLife you get a new feature! and @sschneider you get a new feature!! Numeric() is going to be publicly announced soon.

@MonitoringLife [10:58 AM]
beat me to the oprah gif

@Linds [10:58 AM]
GREAT minds…

@anon85944545 [10:58 AM]
:heart: it

@jwilcox [10:58 AM]
@anon85944545 earlier you asked about MAX(numeric(loadAverageOneMinute)) * 3 > coreCount in a WHERE clause. (edited)

@hross [10:59 AM]
I am :100: percent pleased Oprah made it into this

@jwilcox [11:00 AM]
At the moment NRQL only allows functions in the SELECT statement.

@Linds [11:00 AM]
That is time everyone! I am calling this a very successful Coffee Chat - thanks for coming!

@jwilcox [11:00 AM]
Also - here are some good videos for all the Insights learns: https://learn.newrelic.com/courses/intro_insights/dashboards

@Linds [11:01 AM]
I will be posting a transcript of our time together in discuss.newrelic.com soon! Appreciate all the questions and all the inisight!

@anon85944545 [11:01 AM]
thanks everyone, love these things :slightly_smiling_face:

@twittinger [11:01 AM]
These forum posts are also super helpful with on-boarding: https://discuss.newrelic.com/c/level-up-relic-solutions

@anon85944545 [11:02 AM]
yeah, i remember digging into those getting ready for the APM cert @Tony; all good things and I have a wiki page with all those links for our users right now :smile:

@jbiggley [11:02 AM]
Gah – I blame @anon85944545 for me missing this #coffee-chat

@anon85944545 [11:02 AM]

@jbiggley [11:03 AM]
He cat-fished me into taking a slot for training our run-team on using (wait for it…) New Relic :wink: (edited)

@Linds [11:03 AM]
Well that is a pretty good excuse I guess … :wink:

@anon85944545 [11:03 AM]
I mean, to be fair I’m waiting by the phone for another vendor’s support to call me (which makes NR support look even better than their already stellar track record)

@jbiggley [11:03 AM]
I think so. I’m still giving Zack a hard time about about it though :wink:
I hear I missed NUMERIC() awesomeness as a pre-announcement :heart_eyes:

@hross [11:05 AM]
Oh the places you’ll go with that one @jbiggley

@jbiggley [11:09 AM]

Dang, giphy is on point today all the feels. :raised_hands: