Follow the instructions to get instant observability for your machine learning models in production. You’ll get:
- A guided integration set up for Aporia
- A pre-built dashboard with charts to help you monitor your models
Before you begin, ensure you have a New relic account, or sign up for a free account here (no credit card needed).
You will also need to have an Aporia account, which is free to use. You can get one here.
- Log into Aporia’s console. On the navbar on the left, click on Integrations and choose New Relic.
Aporia allows you to connect alerts generated from Aporia’s monitors to New Relic’s Incident Intelligence engine and the predictions data in order to create a comprehensive monitoring dashboard in New Relic for your models in production.
- Log into your New Relic account, and click on + Add more data.
In the search bar type Aporia (or scroll down to the MLOps Integrations section) and click on the Aporia icon.
Under Prediction data, click the Select or create API key to create a new API key or use an existing one.
- After creating the token, click on the copy symbol
- Then go back to the Aporia dashboard and paste the token under New Relic Insert Token.
- Return to the New Relic dashboard. In case the token doesn’t exist, under Model quality metric alerts, click on the Configure Intelligent correlated alerts link and follow the instructions to create a new token.
- After creating the token, click on the copy symbol, go back to the Aporia dashboard and paste it under New Relic Incident Intelligence Token
- In the Aporia dashboard, click on the Verify Tokens button to verify both tokens are working correctly. Green check marks or red error marks should appear to indicate the status.
- Once everything is set, click on the Save button.
- Return to the New Relic dashboard and click on the See your data button. This will redirect you to a dashboard displaying data reported to Aporia in New Relic.
In order to allow easy filtering of the data, on both the Most Active Models chart and the Most Active Model Versions chart
Click on the … symbol and click edit.
On the right navbar, under User as filter activate Filter the current dashboard and click Save.
The other graphs display statics over the predictions reported
The Model Inferences graph displays the number of unique inferences reported for each model and version.
The Average Numeric Inferences graph displays the average value numeric inferences reported for each model and version.
The Numeric Inferences Heatmaps graph displays heatmaps of the numeric inferences values reported for each model and version.
The Categorical Inferences graph displays the different unique values and their frequencies of categorical predictions reported for each model and version.
When a new alert is detected by Aporia, it will be reported to New Relic’s Incident Intelligence engine. To view these alerts in New Relic, click on Alerts & AI and on the left navbar click on Issues & activity.
On this page you will be able to see the correlated alerts. Clicking on an issue will open a screen with additional data, including all the related activities to the issue and their payloads.
You’ve now successfully integrated Aporia with New Relic. Newly created alerts will now be correlated with your New Relic alerts and you should be able to see data about newly reported predictions.
Get the dashboard, install this quickstart from New Relic I/O.
Aporia allows you to create customized monitors for your ML models in production. With model monitor builder and New Relic, correlate your ML alerts with engineering incidents and get alerts for issues like concept drift, model performance degradation and more.