Java Agent Troubleshooting Framework Logs in Context

Dropwizard v1.3 extension

General

  • Review the Compatibility and Requirements for setting up Logs in Context for the Java agent and Dropwizard:
    • Java agent 5.6.0 or higher
    • JVM argument -javaagent enabled on the Java agent.
    • Dropwizard 1.3 package installed and working on the application, with the original Dropwizard appenders and logging factory.
      • Make sure Dropwizards default settings are working and logging as expected before implementing New Relic’s Dropwizard logging extension.
  • Ensure you have distributed tracing enabled for the Java agent.
  • Check your application dependencies and make sure our Dropwizard extension is included.
  • Review your Dropwizard config file for the following:
    • Check that you are using type: newrelic-console or type: newrelic-file in place of the default type: console or type: file provided by Dropwizard.
    • Check your layout section and make sure you are using type: newrelic-json.
    • We also recommend reviewing Dropwizard’s logging documentation:
  • Once you have configured the enricher it is a good idea to examine your logs to check that they are formatted as JSON and contain the following fields:
    • “entity.name”
      • Should be the application name set in APM for the service you’ve configured Logs in Context for.
    • “entity.type”
      • Should be the string “SERVICE”
    • “entity.guid”
      • The ID of the entity.
    • “trace.id"
      • Will only be present if distributed tracing is enabled and the thread writing the log has context of an existing Transaction.
    • "span.id”
      • Will only be present if distributed tracing is enabled and the thread writing the log has context of an existing Transaction.
    • “hostname”
      • This should be the hostname gathered by the APM agent.

Troubleshooting

  • Make sure your APM language agent is installed and set up correctly.

  • Make sure that you have Distributed Tracing configured correctly for your application. If you do not you will not see a span.id and trace.id in your log metadata. However, a missing Trace or Span ID could also indicate that the application just wasn’t aware of the Transaction when it wrote the logs so it is best to confirm the configuration is enabled in the “Environment” page of the UI or directly with the Java Agent config file.

  • If you aren’t seeing logs in the Logs UI. Ensure that you have installed and configured a New Relic plugin for a compatible log forwarder:

  • Check your Logs UI in New Relic and query using the the following:

    • has: span.id has:trace.id
  • Check out our source code here:

  • Check your configuration against our examples on Github:

java.util.logging extension

General

  • Review the Compatibility and Requirements for setting up Logs in Context for the Java agent and Dropwizard:
  • Ensure you have distributed tracing enabled for the Java agent.
  • Review your dependencies and logging properties file:
    • Check your application dependencies and make sure our java.util.logging extension is included.
    • NewRelicMemoryHandler should be set as the root handler in your logging properties file.
    • The original root handler should be set as the target for NewRelicMemoryHandler.
    • NewRelicFormatter should be the set via the formatter property on the original root handler.
    • Review steps one through five in our public documentation for examples on the above configuration, pay special attention to the “Tip” boxes:
  • Once you have configured the enricher it is a good idea to examine your logs to check that they are formatted as JSON and contain the following fields:
    • “entity.name”
      • Should be the application name set in APM for the service you’ve configured Logs in Context for.
    • “entity.type”
      • Should be the string “SERVICE”
    • “entity.guid”
      • The ID of the entity.
    • “trace.id"
      • Will only be present if distributed tracing is enabled and the thread writing the log has context of an existing Transaction.
    • "span.id”
      • Will only be present if distributed tracing is enabled and the thread writing the log has context of an existing Transaction.
    • “hostname”
      • This should be the hostname gathered by the APM agent.

Troubleshooting

Logback extension

General

  • Review the Compatibility and Requirements for setting up Logs in Context for the Java agent and Logback:
    • Java agent 5.6.0 or higher
    • Logback 1.2.0 or higher installed and working on the application.
  • Ensure you have distributed tracing enabled for the Java agent.
  • Review your dependencies and logging configuration xml file:
    • Make sure our logback extension is included correctly.
    • The appenders that you write log events with should have their elements replaced with the New Relic encoder:
    • NewRelicAsyncAppender should be the first level appender in your configuration and references your existing appenders as its children by using
    • NewRelicAsyncAppender should be referenced as the primary appender in your configuration’s section.
      • It’s important that the NewRelicAsyncAppender be the first appender to see the log message. List any other appenders after the NewRelicAsyncAppender in the list.
    • Review steps one through four in our public documentation for examples on the above configuration:
  • Once you have configured the enricher it is a good idea to examine your logs to check that they are formatted as JSON and contain the following fields:
    • “entity.name”
      • Should be the application name set in APM for the service you’ve configured Logs in Context for.
    • “entity.type”
      • Should be the string “SERVICE”
    • “entity.guid”
      • The ID of the entity.
    • “trace.id"
      • Will only be present if distributed tracing is enabled and the thread writing the log has context of an existing Transaction.
    • "span.id”
      • Will only be present if distributed tracing is enabled and the thread writing the log has context of an existing Transaction.
    • “hostname”
      • This should be the hostname gathered by the APM agent.

Troubleshooting

  • Make sure your APM language agent is installed and set up correctly.

  • Make sure that you have Distributed Tracing configured correctly for your application. If you do not you will not see a span.id and trace.id in your log metadata.

  • If you aren’t seeing logs in the Logs UI. Ensure that you have installed and configured a New Relic plugin for a compatible log forwarder:

  • Check your Logs UI in New Relic and query using the the following:

    • has: span.id has:trace.id
  • Check out our source code here:

  • Check your configuration against our examples on Github:

  • Reference Logback’s documentation here:

  • Logback can be set to “debug mode” in order to output more detailed information about it’s configuration/operation by changing the line of the customers XML logback configuration file to

Log4j 1.x extension

General

  • Review the Compatibility and Requirements for setting up Logs in Context for the Java agent and Log4j 1.x:
  • Ensure you have distributed tracing enabled for the Java agent.
  • Review your dependencies and logging configuration xml file:
    • Check that our log4j 1.x extension is included correctly.
    • The appenders that you write log events with should have their elements replaced with the New Relic’s layout:
    • NewRelicAsyncAppender should be the first level appender in your configuration and references your existing appenders as its children by using
    • NewRelicAsyncAppender should be referenced as the primary appender in your configuration’s section.
      • It’s important that the NewRelicAsyncAppender be the first appender to see the log message. List any other appenders after the NewRelicAsyncAppender in the list.
    • Review steps one through four in our public documentation for examples on the above configuration:
  • Once you have configured the enricher it is a good idea to examine your logs to check that they are formatted as JSON and contain the following fields:
    • “entity.name”
      • Should be the application name set in APM for the service you’ve configured Logs in Context for.
    • “entity.type”
      • Should be the string “SERVICE”
    • “entity.guid”
      • The ID of the entity.
    • “trace.id"
      • Will only be present if distributed tracing is enabled and the thread writing the log has context of an existing Transaction.
    • "span.id”
      • Will only be present if distributed tracing is enabled and the thread writing the log has context of an existing Transaction.
    • “hostname”
      • This should be the hostname gathered by the APM agent.

Troubleshooting

Log4j 2.x extension

General

  • Review the Compatibility and Requirements for setting up Logs in Context for the Java agent and Log4j 2.x:
  • Ensure you have distributed tracing enabled for the Java agent.
  • Review your dependencies and logging configuration xml file:
    • Check that our log4j 2.x extension is included correctly.
    • Your configuration element should reference the New Relic log4j package:
      • packages=“com.newrelic.logging.log4j2”
      • If using a properties file, make sure the above package is referenced there.
    • Any appenders you write log events with should contain .
      • If using a properties file, ...layout.type should set to NewRelicLayout
    • Reference the above appender in your section with .
      • If using a properties file you will to reference the appender there.
    • Review steps one through four in our public documentation for examples on the above configuration:
  • Once you have configured the enricher it is a good idea to examine your logs to check that they are formatted as JSON and contain the following fields:
    • “entity.name”
      • Should be the application name set in APM for the service you’ve configured Logs in Context for.
    • “entity.type”
      • Should be the string “SERVICE”
    • “entity.guid”
      • The ID of the entity.
    • “trace.id"
      • Will only be present if distributed tracing is enabled and the thread writing the log has context of an existing Transaction.
    • "span.id”
      • Will only be present if distributed tracing is enabled and the thread writing the log has context of an existing Transaction.
    • “hostname”
      • This should be the hostname gathered by the APM agent.

Troubleshooting

  • Make sure your APM language agent is installed and set up correctly.

  • Make sure that you have Distributed Tracing configured correctly for your application. If you do not you will not see a span.id and trace.id in your log metadata.

  • If you aren’t seeing logs in the Logs UI. Ensure that you have installed and configured a New Relic plugin for a compatible log forwarder:

  • Check your Logs UI in New Relic and query using the the following:

    • has: span.id has:trace.id
  • Check out our source code here:

  • Check your configuration against our examples on Github:

  • Reference Apache’s docs for log4j 2.x:

  • Log4j2 can be configured to output debug information about it’s configuration. To do this you just need to as status=”debug” to the <Configuration… line of the XML. For example: <Configuration status=“debug”

Spring and Spring Boot

General

  • Review our documentation and determine which logger your version of Spring and Spring Boot are using.
  • Navigate to the troubleshooting framework for the logger you are using.