Thread Profile JSON handling and possible bug

Hi!

I’ve developed a script (here: https://gist.github.com/DJRickyB/ee0bfd732af64aa44ad8421fa0d9d5ca) that parses thread profile JSON and outputs collapsed txt compatible with FlameGraphs (https://github.com/brendangregg/FlameGraph) (… actually it outputs two files, one for what i think is CPU and one for what i think is WAIT).

(a) can anyone validate assumptions being made in this script or point me to a schema for the data in this JSON file?

(b) i noticed even after for accounting for stacks that may have not been as deep as others that i don’t seem to find anything at the top other than the furthest leaves of the stack. to give an example, if I were to use the following as my collapsed.txt and run flamegraph.pl collapsed.txt > test.svg the following occurs:

collapsed.txt:

method1;method2 5
method1 10

test.svg:

note the red area, which indicates time spent in method1 and NOT in method2. it seems unlikely that this would not occur in the real world so i wonder if a given branch’s values are ONLY the sum of its leaves, rather than its own count. could be interesting that a bunch of execution times are getting dropped because there are times when the stack goes deeper

HI, @rick.boyd: I have no information about the schema of thread profiler JSON, but I don’t think it includes time spent in each method. The percentages that appear when you view the profile in the UI are the percentage of stack trace samples that include a particular method. Since a sample may include hundreds of methods, the percentages likely add up to greater than 100%. More info here: https://blog.newrelic.com/product-news/apm-thread-profiler/.

Right, it’s a sampler, I understand it would not contain time spent. In question is whether the values at a branch point are ignoring samples where that branch was the top of the stack. Evidence says it is

aha! after some more debugging i was able to find cases where the substack sample count was accounted for separately, so no worries on #2. can someone confirm that in the JSON you can export from the UI the schema of a given branch is:

branch tuple:
element 0: [Class, Method, LineNumber]
element 1: CPU Time
element 2: Wait Time
element 3: [children…]

1 Like

@rick.boyd - What a great thread! Thank you so much for sharing everything you learned with the community. Gold star! :star2: