The week of November 5, 2018, we will be making some changes to the Insights API. We are upgrading the supporting platform in order to greatly improve the reliability, scalability, and performance of this API. For the most part, the migration of this API to the new platform will be totally transparent to you. However there are a few operational changes we want you to be aware of, and some existing limits that we want to remind you of.
We are also taking this opportunity to update the naming of this API. Instead of referring to this API endpoint as the “Insights API”, we will simply refer to it as the “Event API”. This is the term that we will use in this post, and we will be updating our docs and public content over the next few weeks.
What you should know…
1. Updated documentation.
We have updated the docs to reflect all of the new changes. See : “Send custom events with the Event API”
2. Change in when we parse and validate content.
We are delaying content parsing until after we receive the content. We refer to this as being an asynchronous API. The role of the Event API is to accept the message payload from the HTTP response, reliably queue it, and return a successful response code as quickly as possible.
We will still synchronously validate all aspects of the POST request, including license keys, required headers, and payload size. It is just the content parsing and validation that will happen after we have returned a successful HTTP:200 OK response code. If parsing errors occur, we will create an NrIntegrationError event in your account that captures the details. Please review the documentation to see how you can query for these events, create alerts to monitor for them, and search for errors related to specific POST responses.
3. Reminder: There is a 1MB limit on the size of the POST payload.
While the 1MB payload size limit is documented, we have not always strictly enforced it. With this upgrade, this limit will be strictly enforced. HTTP POSTs with a payload larger than 1MB will be rejected with an HTTP 413 (Content too large) response code. We strongly encourage you to compress your content before POSTing it to the API.
4. Default API rate limits
In order to ensure the reliability and availability of our multi-tenant platform, it is critical that this API enforces rate limits across all accounts. We must ensure that extreme, unexpected traffic spikes do not negatively affect the platform and degrade the service for all of our customers. The new API platform will automatically enforce a maximum default rate of 100,000 requests per minute.
If your API usage exceeds 100k POSTs in a 1 minute window, we will reject subsequent API requests with a 429 response code for the remainder of the 1 minute window. At the end of the 1 minute window, we will reset the counters and allow traffic to resume until the threshold is triggered again.
These rate limits are intended to be an upper threshold that you should not hit under normal usage scenarios… If you experience an unexpected number of 429 responses, please contact Support to discuss your needs.