AWS Kinesis Firehose Troubleshooting Framework Log Forwarding

This guide is intended for help troubleshooting logs from the configuration and data collection end. For example: missing logs and/or log messages, unparsed logs, config questions etc. If the issue resides in the UI then there is a separate troubleshooting guide.

Before reading further:

  • Ensure that you have installed and configured a compatible Log Forwarder. You can find a list of the different log forwarding options and how to install New Relic’s various plugins in our documentation here. Troubleshooting for individual forwarders can be found below.
  • Logs in Context is different from Log Forwarding. Setting up Logs in Context for your application simply adjusts your application’s logger to format logs as JSON with New Relic’s logging metadata. This metadata establishes context for your application logs and other features of your New Relic APM agent. Check out our Configure logs in context with APM agents doc for more detailed information.

General

Troubleshooting

  • Check for errors using the following NRQL query:
    • For example: SELECT * FROM NrIntegrationError SINCE 24 hours AGO
  • You’re seeing error 413 payload too large. There is a payload size limit for New Relic’s firehose endpoint. The issue results from different interpretations of the 1MB buffer limit imposed on the AWS side.
    • As a workaround you can try enabling gzip encoding in the firehose data delivery stream config. If that doesn’t resolve the issue, then you should try reducing the buffer size.