Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Java Agent: Distributed Trace payload is empty!

java
javaagent

#1

I am using the Java Agent API to demonstrate Distributed Tracing between two services using AWS Simple Queueing Service (SQS).

When I try to create a distributed trace payload, the variable is empty! There is no clue in the logs as to what is going on. See my code snippet below

@Trace(dispatcher = true)
private void addMessageToQueue(String message) {
    // Get a distributed trace payload to include with the SQS message so we can see a trace
    String dtPayload = NewRelic.getAgent().getTransaction().createDistributedTracePayload().text();

    // Create a new message attribute to include the distributed trace payload
    System.out.println("distributed trace payload " + dtPayload);
    MessageAttributeValue msgAttribute = new MessageAttributeValue().withStringValue(dtPayload);
    Map<String, MessageAttributeValue> msgAttrMap = new HashMap<>();
    msgAttrMap.put("dtPayload", msgAttribute);

    SendMessageRequest send_msg_request = new SendMessageRequest()
            .withQueueUrl(queueUrl)
            .withMessageAttributes(msgAttrMap)
            .withMessageBody(message)
            .withDelaySeconds(5);

    sqs.sendMessage(send_msg_request);
}

I am able to verify that the agent is working and I can do things like read the Agent config and get the transaction name programmatically.

  • Java Agent Version: 4.10.0

#2

Hi @danielfitzgerald

I was able to duplicate this issue, however…it was caused by distribute_tracing being disabled. Could you verify in your newrelic.yml that enabled: true is configured for distributed_tracing ?

Cheers!
Matt


#3

Hi @MrMatt ,

I can confirm distributed_tracing = enabled in my yml config. I can confirm this by adding the following line to my code and the console output is ‘true’

System.out.println(NewRelic.getAgent().getConfig().getValue("distributed_tracing.enabled").toString());

I even tried adding the below flag to the VM options but it doesn’t work

-Dnewrelic.config.distributed_tracing.enabled

#4

Thanks @danielfitzgerald, I figured you had it enabled but had to check :slight_smile: I checked our code for this call and it looks like we’ll log any exceptions that occur if you set log_level: finest in your configuration. Could you set that log_level and look for one of those following lines:

"Created distributed trace payload: {0} for transaction: {1}", payload, this
OR
Unable to create distributed trace payload

The “Unable…” line should be preceded by the exception that occurred.

If you find those log lines and are able to upload them to this post I can definitely take a look!


#5

HI @MrMatt

In my log it says:

com.newrelic FINER: Not creating distributed trace payload due to null accountId.

#6

Hey @danielfitzgerald I’m going to move this discussion into a ticket since we’re going to need to request some private information, you should get an email about the ticket shortly. For anybody who comes across this discussion I’ll respond back here once we find the solution!


#7

Hi,

This is an update on how this particular issue was solved. The customer added this configuration to the newrelic.yml file:

sync_startup: true

And then agent was able to connect early enough to catch the call to Distributed Tracing. We believe this error ““The message attribute ‘dtPayload’ must contain non-empty message attribute type.”” in the New Relic logs was showing up because the call was happening right after the application had already started.

You can read more about this configuration setting here:
https://docs.newrelic.com/docs/agents/java-agent/configuration/java-agent-configuration-config-file#cfg-sync_startup