2018 Explorers Hub Survey Results

Hopefully it’s obvious to you all, but the Community Team spends a LOT of time talking to customers in the Explorers Hub. We know a lot about our customers collectively, and have made friends with some real characters. However, we don’t want to assume that just because we spend a lot of time with our customers doesn’t mean that we know everything they think about us and our work.

As a first step to getting to know our community better, we published a survey. Over the coming months, we have other questions we will want to dig into, and we may start discussions in the community, or ask to interview our community members as a follow up.

Our survey questions were designed to help us understand if some of the changes we’ve made in the community over the last year have been well-received by our Explorers. We have focused our strategy on helping new Explorers get oriented in the community and engage more. As part of that, we reorganized categories, implemented badges, and have been using email to onboard new Explorers and let them know about campaigns in the community.

How is all that being received? Let’s find out.

How do we keep Explorers coming back for more?

There are multiple reasons why a customer would engage in a support community. Some are intrinsic (like wanting to be seen by peers as an expert) and some are extrinsic (like getting a free t-shirt). While different people may find various rewards more appealing, individuals may also find different rewards appealing at various times in their New Relic journey.

The Community program needs to balance the rewards we offer to appeal to this wide variety of experiences. So we asked:

The top two answers are really straight-forward: our customers want answers quickly, and they want to be engaged in pre-release opportunities. In short, they are here to do business, and as much as we want them to engage socially, helping them better use New Relic has to be at the center of every interaction we plan on the community team.

Side note - we JUST re-introduced badging in the Explorers Hub this spring. We got a fair number of comments along the lines of “what’s this badging thing you mentioned,” so we’re actually pretty pleased with how positively this ranked at this point in time.

But - SURPRISE - the answer is actually a T-shirt.

While our customers told us that they are in the community to get down to business, we also learned that really, that business should come with a T-shirt attached! We reward people for participating in the community in a variety of ways, so we dug a little deeper into the engagement question by asking:

It looks like the Community team is going to stay close to the staff at our local post office. Again, we were happy to see how well the new badging system scored as well. We don’t plan to expand these significantly, but we’ll continue to educate customers about their existence and how to earn them. It was also interesting to note that the individualized efforts of messaging and at-mentioning users were of less interest (though not none). These are labor-intensive moderation tactics that we can reserve for the places where they will have the most impact or meaning.

The Emails are All Right

We added a new dimension to our community program this year - email! We routinely email our customers about the campaigns we are running and other community business. It’s been great for our engagement metrics. Many more people are participating in campaigns and generally being part of the community. We keep our sends to a minimum (never more than once in a week generally), and we coordinate large campaigns with our friends on other New Relic teams. While our unsubscribe rate has been low, we’re always nervous that we are sending too much. So we asked:

We’re probably hitting about the right cadence, and it’s probably the right medium. We know this because next we asked customers:

Email had the strongest preference at the weekly cadence, while customers prefer at-mentions and Private Messages in the platform for more frequent messaging needs. Interestingly, they were not very interested in Twitter.

Show your work. How’d you get that answer?

You may have noticed that the Explorers Hub is organized a lot like New Relic itself. We have categories and sub-categories for every. single. product. We replicate that structure for different kinds of content too (support questions vs. Level Up posts vs. Feature Ideas, for example). We also tag content kind of obsessively. We do all that work, even though 70% of the traffic to the Explorers Hub comes from Google and lands our customer on a specific page - not our home page.

Add in the fact that New Relic never stops innovating so our products change rapidly, and you can see that all this work to keep content neatly organized is kind of a lot. We wanted to know if our community members valued that work at all, so we asked:

No surprises about the top spot - Google is the top way people find information on the Explorers Hub. We did find it surprising that so many people use the search bar within the community, however. We are generally concerned that it does not return great results, but it must be providing decent value.

But what about all those categories? Are the helpful though? So we dug a little deeper on this theme and asked:

The averages across all the options were very even, which indicates to us that while none of these options are particularly valuable to our customers, none of them aren’t valuable either. Translation: ¯_(ツ)_/¯.

So we dug a little deeper and asked:

Only about 10% of customers use categories to navigate through the Explorers Hub. That’s… not a lot. At the very least, I think we need to talk to some customers to understand how and why they use the category structure.

Who are the Explorers?
Lastly, we wanted to know more about the kind of customer that is engaged in the Community. Knowing who our Explorers are will help us better shape our content and programming. To get an idea of where Explorers are in their New Relic journey, we asked:

Half of our customer respondents are in their first year of using New Relic. This is completely expected. The primary function of the Explorers Hub is to provide support, and most of our customers need the most support as they first engage with one of our products. This closely mirrors what we see with customer tickets. These customers bring value to the community by reading a lot of content, liking that content when it is helpful, asking good questions, and marking the answers to their questions.

It’s also good to see that 50% of Explorers have been using the product for a year or more. These customers may be brushing up their skills, doing something new, or might be adopting a new product within New Relic. The opportunity for the Community team with these customers is to help them see themselves as experts who can (and should!) share the knowledge they have with the other half of our community.

What are we doing with this data?

As I mentioned, this is the first step in a journey to better understand our customers’ needs. Why do you turn to the community, what works for you, and what do you wish we were better at are all questions we want to discuss more with you. So largely, were using this data as a launching off point for more conversations with our Explorers.

We’re also taking some action though. Based on this feedback, we’ll be taking a closer look at how content is organized in the community, and how we can deliver better on what our customers really want - fast, thorough, and accurate answers to their burning questions.