[Java] Distributed Tracing with Java Agent 7.4.0, Scala 2.13, Play 2.8, and Akka HTTP 10.2.7 not working as expected

Hi, we’re running multiple Play applications using Scala 2.13 and Play 2.8.11 with Akka HTTP 10.2.7 (with HTTP2 support) as the backend, and we’ve noticed that distributed tracing doesn’t seem to work: requests between our services are reflected only as “external service” in the service map and the trace itself (as opposed to referring to the correct service). All services are have the java agent running (they share the same docker image and build config, with service name + license key being passed in as environment variables). We’re also aware of the shortcomings around instrumentation for Play WS under Scala 2.13, and have manually added instrumentation for it.

I looked into the logs, it does store the TracedWeaveInstrumentation for all classes from com.newrelic.instrumentation.akka-http-2.13_10.1.8 and com.newrelic.instrumentation.akka-http-core-2.13_10.1.8, but when a request comes in, none of those classes are getting instrumented, and so it doesn’t seem like anything’s accepting the trace headers.

As I was writing this up, I did notice a couple interesting logs, it seems like the instrumentation first sets the akka-http port to 8558, and then attempts to set it to 9000 (the port we’re actually using) but ignores it because it’s already set to 8558. It looks like 8558 is the default port for akka management, would this then imply that what’s being instrumented is the akka management endpoints, and not the Play endpoints? And if that’s the case, how would we go about resolving this?

Any help would be greatly appreciated. I’ll be away next week and so unlikely to respond for clarification or more details, but will be back in the new year. Thanks!

Hi @nemond good news : with our 7.5.0 release, you should see better instrumentation support for Play WS and Scala 2.13.

We’re not sure whether this will resolve your issues with Distributed Tracing though but we’d encourage you to get the next release in a non-prod environment and give it a go!

Regarding our instrumentation and port usage; we don’t have any logic in our agent around ports, so it’s more likely that you were just running into a limitation in the supported frameworks for that version of the agent.