Log API Troubleshooting Framework Log Forwarding

This guide is intended for help troubleshooting logs from the configuration and data collection end. For example: missing logs and/or log messages, unparsed logs, config questions etc. If the issue resides in the UI then there is a separate troubleshooting guide.

Before reading further:

  • Ensure that you have installed and configured a compatible Log Forwarder. You can find a list of the different log forwarding options and how to install New Relic’s various plugins in our documentation here. Troubleshooting for individual forwarders can be found below.
  • Logs in Context is different from Log Forwarding. Setting up Logs in Context for your application simply adjusts your application’s logger to format logs as JSON with New Relic’s logging metadata. This metadata establishes context for your application logs and other features of your New Relic APM agent. Check out our Configure logs in context with APM agents doc for more detailed information.

General

  • Make sure that you have set your license key or api key in your request headers otherwise the log data you are wanting to send to our endpoint will not know what account to associate the data with.
    • See our required HTTP headers in our docs here
  • Ensure the logs you are sending are in valid JSON format. We accept any valid JSON payload. The payload must be encoded as UTF-8.
    • Please review limitations on what logs can be sent via the API here: Limits and restricted characters
    • Log management does not support white space in attribute names. For example, {“Sample Attribute”: “Value”} would cause errors.
  • Double check that you are using the correct endpoint for your region.

Troubleshooting

  • Sometimes you’ll see a 202 success code when sending logs to New Relic via the API but won’t see logs in the UI. If that’s the case, it’s best to check for errors using the following NRQL query:
    • For example: SELECT * FROM NrIntegrationError SINCE 24 hours AGO
  • When the maximum request rate limit is exceeded for an account or when the maximum Log JSON byte limit is exceeded, the New Relic Log API returns a 429 response for the remainder of the minute. This response includes a Retry-After header indicating how long to wait in seconds before resubmitting or sending new data.
    • The maximum rate for HTTP requests sent to the Log API is 300,000 requests per minute.
      • If you exceed this then you will need to either reduce the number of data points you are sending, or request a rate limit change. Subsequent subscription changes do not impact modified rate limits. If an account change impacts your rate limit, you must notify us to adjust your rate limit. To request rate limit changes, contact your New Relic account representative
    • The maximum rate of uncompressed Log JSON bytes sent to the Log API is 10 GB per minute
      • If you exceed this then try to reduce the amount of log data you are sending, or spread it out over a larger period of time.