Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Raw async traces in thread-profiler, no option to select transaction


#1

.NET Agent Question Template

  • Please provide a permalink (I’d have to get permission) .NET agent version:
    NewRelic.Agent.Core.dll: 8.15.455.0+sha.7bdf4e8db.branch.HEAD

  • Are you using .NET framework or .NET Core?
    .NET Framework 4.6

  • Please let us know which operating system you have questions about:
    Windows

Troubleshooting

  • Describe what you are seeing:
    In thread profiler, raw async traces, no options to filter by transaction.

  • How does that differ from what you are expecting to see?
    I’d expect to be able to choose which transaction I’m looking at, and for the traces to have the compiler bits taken out, so that it makes sense in terms of the original source code e.g. https://github.com/benaadams/Ben.Demystifier

Does the thread profiler benefit from the improved traces available in .NET Core? i.e. https://www.ageofascent.com/2018/01/26/stack-trace-for-exceptions-in-dotnet-core-2.1/

As it stands the tool seems nearly unusable except in the case where the server is falling over and there is an obvious problem. To figure out the reason for slow transactions it’s nearly impossible to find an instance of the transaction in question, let alone relate that to the original application code.

We’re not deploying any symbols with our app e.g. .pdb, would this help the agent?


#2

Hi there @dave.higgins,

Great questions! I wanted to provide some more background about the thread profiler and how it can be beneficial in diagnosing issues — especially for issues related to CPU usage.

The thread profiler can be useful as far as seeing a percentage breakdown of how much time a thread is spending on any particular method. When you run the thread profiler, it takes a snapshot of your application threads every 100ms for the life of a profiling instance. After that’s done, it compares all of the snapshots and shows you, based on percentages, how much time is spent in each of those methods.

One of the more helpful features we have in the thread profiler is the “Filter Outliers” checkbox. This will go through and find most of the unused methods and remove the whole tree from being displayed, which will un-clutter the interface significantly.

I don’t believe deploying the app with symbols would have any effect on the information that is collected, but I do recommend that you make sure to rigorously exercise the app while the thread profiler is running in terms of sending traffic that way.

You might have better luck, though, looking at the application’s transaction traces:

These provide timing information for each method that is instrumented and they tend to be more useful for troubleshooting issues with slowness. They are also searchable.

To get the agent to generate more transaction traces for slow transactions, you can lower the transactionThreshold to a value in ms:

Let me know of any other questions!


#3

Hi @ntierney,

The slow transactions do not include any calls into the application code, which is necessary to diagnose the problem without attempting to reproduce the problem locally. We can’t permanently turn on instrumentation for MyApplication.* on a production application either.

Thread profiler on the other hand offers the opportunity to sample/instrument the application temporarily, with full traces into application code - perfect. Unfortunately the lack of unwinding of async etc leads to a mixing of transactions across threads - I suspect this is why the grouping by transaction available for other newrelic agents is not available for ASP.NET. As I’ve mentioned, this is nearly impossible to read. Likely even if it were grouped by transaction, it would still be very difficult to read. That’s why I’ve mentioned the links in my previous post.

So there’s a couple of things I can think of that might help:

  1. Turning on full instrumentation for an application instance for a duration (via the newrelic web client?), collecting slow transactions and being able to filter them by those that contain fully instrumented application code calls.

  2. Applying custom analyzers to the output of the thread profiler to provide custom groups and transformations of the traces. Either of the links from my previous post could be used (or perhaps newrelic will add this to their service by default).

Are either of those things a possibility?

Thanks!


#4

Hi, @dave.higgins: Both of your suggestions are interesting feature ideas, but they are not currently possible.

As a possible workaround, if you append “.json” to your thread profiler URL, you can download the raw JSON file, which you may be able to view or filter in a different tool: https://rpm.eu.newrelic.com/accounts/{your_account}/applications/{your_application}/profiles/{profile_id}.json