I would like to be able to set days of retention by eventType within the Event data retention management page for Insights Custom Events.
This would allow me to have long-term retention on some events (6-12 months), while purging events that I only need for a shorter period (24 hours to 7 days, for example).
I create a lot of custom events with New Relic Insights, using the Insert API. These events are extremeley useful to my business, but some of them are only needed for a short period, while I’d like to keep or create other events that are retained for longer, so that I can query and create dashboards that go back several months.
Due to the way Insights licensing & usage works, my “bucket” fills up with events containing data that could be aggregated into a much smaller number of events, and retained for longer.
Feature Request Details
If I could set custom retention by event type, and have some events purged, then I could keep the total number of events within the licensed amount, and while also having longer retention for other events where necessary.
In order for this to be most useful, I would want to create additional “aggregated” custom events from the higher quantiy / more frequent timeseries events, to allow longer retention of important information, but at less detail. These would be retained for a longer period.
If I could use New Relic itself to create the aggregated events automatically, that would be even better, but it wouldn’t be hard to query the Insights API, and then just feed the results back in as new Custome Events.
It would appear that this could be achieved via a “hack” by using Sub-Accounts: Event data retention | New Relic Documentation
It looks like a sub-account could be used for each set of Insights Custom Events that require a different retention. However, this is obviously a “sledgehammer to crack a nut” scenario.
An additional feature, or possibly even an alternative, would be to specify custom aggregation for events. This is apparently already built-in to APM, Browser, and Mobile, in some fashion. I forsee the need to aggregate in other ways as well (sum, for example), and to specific the attribute / facets that are aggregated.
New Relic Docs
- Use the Event API to report custom events | New Relic Documentation
- Event data retention | New Relic Documentation
New Relic Edit
- I want this too
- I have more info to share (reply below)
- I have a solution for this
We take feature ideas seriously and our product managers review every one when plotting their roadmaps. However, there is no guarantee this feature will be implemented. This post ensures the idea is put on the table and discussed though. So please vote and share your extra details with our team.