New Relic released an important new capability for APM language agents that integrates application logs with APM. When you upgrade to the latest versions of the Ruby, Java, and .NET agents, the APM agent will add entity context and forward the logs directly to New Relic, eliminating the need for third-party applications to be installed or configured.
Starting May 3rd, 2022, you get the following three key benefits when you update to these new APM agents:
- Eliminate manual log configuration and maintenance necessary to collect and forward logs to New Relic when using Java, .NET, and Ruby APM agents.
- Minimize context-switching by viewing application logs in context with APM metrics, traces, and events.
- Troubleshoot issues faster by accessing logs inside the APM with enhanced UI that help you troubleshoot issues faster.
When collecting logs, it is critical to ensure security, compliance, and control. This is why we are including robust support for the following:
- Opt-out anytime: Turn off automatic forwarding at any point by configuring your agent or using New Relic’s data management hub.
- Ingest control: Use in-agent log sampling to manage ingested volume and avoid duplicating ingest and get 3X more value than alternate log management solutions.
- Compliance: Log collection is disabled by default for HIPAA, PCI-enabled accounts, and accounts where High Security Mode is in use, even after you upgrade the agent.
- Data security: Mask, obfuscate, and prevent sending PII, PHI, or any other sensitive data via customizable security configurations.
Note: Application logs in APM is only available after an APM agent update. Until you upgrade, you will see no changes in your account.
APM Agents with logs in context
Logs are a critical telemetry type for observability and now, with automatic logs in context available for APM agents, you can immediately view relevant logs associated with other application telemetry data so you can find and fix issues faster.
Currently supported APM languages and frameworks:
- Java
- Log4j2
- Logback
- .NET
- Log4net
- Serilog
- Microsoft.Extensions.Logging
- Ruby standard library logger
What you’ll need
Before you begin, ensure you have a New Relic account, or sign up for a free account here (no credit card needed).
You will also need to have the latest version of the Java, Ruby, and .NET agents:
Avoid duplicating log data
In certain scenarios, you might end up with “duplicate” logs being collected. This could be because the same log information is being sent to New Relic twice (for example, if you use a third-party log forwarder.) To avoid duplicating log data, consult this documentation.
What about sensitive log data being sent to New Relic
You control what log data is sent to New Relic, and collecting logs using the APM agent is no different. Be sure to follow your organization’s security guidelines to mask, obfuscate, or prevent sending PII, PHI, or any other sensitive data. To learn more about New Relic’s log management security and privacy click here.
Please note that no logs of any kind will be collected if High Security Mode is enabled on the agent, even after the agent is upgraded. It is also possible to configure drop filters to help prevent sensitive data from being stored in New Relic.
Turning automatic log forwarding off
The feature will be on by default for new installations or upgrades starting May 3, 2022. If you want to halt logs ingest:
- You can configure the agent config file locally on a machine to disable it.
- You can disable logs ingest for APM agents at the account level with a toggle in the New Relic data management hub1. The toggle can be flipped before ever setting up an APM agent that forwards log data. Shutting off the logs at the account level is an all or nothing choice. There is no way to allow-list specific agents to send data via controls within the New Relic UI. Disabling logs via UI may cause a higher-level of resource utilization. Please disable logs via the agent config file if you have resource-sensitive systems.
Support
For more information about obfuscating sensitive data in logs, read these docs or contact Support.