So, you added some custom instrumentation for your .NET application with a custom XML file, but you’re not seeing the results you expected. This could mean a lot of things, but the first thing to do is make sure there aren’t any problems with the XML file itself.
If you haven’t created your custom instrumentation file yet, start here.
For the rest of you, let’s get started.
Is your XML in the right directory?
Custom XMLs should be placed in the
.NET Framework agents installed on Windows servers use
C:\ProgramData\New Relic\.NET Agent\extensions(make sure you do not use the
extensionsfolder in the
.NET Core agents, or any agent installed with the
Agent.ApiNuGet package use the
extensionsfolder inside the directory where the agent was installed.
If you’re not sure if you’re in the right directory, look for existing
.xml files in the same directory with names like
NewRelic.Providers.Wrapper.*.xml. If you see files like that, you’re in the right place.
Is your XML valid?
The easiest way to make sure your custom instrumentation file is valid (free of structural issues) is to validate it against the supplied
extensions.xsd schema provided with the agent. This file is located inside the agent’s
extensions folder (the same folder where you place your custom
There are many tools for validating an xml against its schema (
.xsd). This post isn’t intended to be a tutorial on that. If you don’t have a dedicated tool installed on your machine, there are free “xml to xsd validator” sites online that can help.
If your xml does not validate against the xsd file, it will not be parsed by the agent and your instrumentation will not be applied.
Lets get some logs!
Now we’ll generate some profiler logs that will allow us to see exactly what the agent is doing with your xml file when your application runs.
Open the global
newrelic.config file for your agent.
- Framework agents installed with the msi, you’ll find it at
C:\ProgramData\New Relic\.NET Agent\.
- All other agents (.NET Core or installed via the
Agent.ApiNuget package) it will be in the directory where the agent was installed.
If you are using localized config files in the root folder of your app, do not use those ones for this process. Only use the global newrelic.config file.
In that file, find the line
`<log level="info" />`
and change it to
`<log level="finest" />`
Save the file, then restart your application and exercise it in a way that calls the methods you’re trying to instrument in your xml file.
While your application is running, take note of the process ID (PID) your application is executing under (if your application is running in IIS, note the PID of the
w3wp.exe process hosting your app).
The logs you generated should be located at:
- .NET Framework agents -
C:\ProgramData\New Relic\.NET Agent\Logs
- .NET Core agents or agents installed with NuGet - In the
logsfolder inside the directory where the agent was installed.
Note, if you don’t see logs being generated in that directory, stop and go check out this post.
Get the right log file
Grab the PID your application was running under and look for a log called
What to look for in the logs
Is the agent seeing the method being called?
In that log file, search for your method name and you should see a line that fits this pattern…
[Trace] <<DateTime>> Possibly instrumenting: (Module: <<path/to/your/module>>, AppDomain: <your domain>>)[assembly]assembly.class.method(parameters)
If you don’t see that line in the file, your method was not called in the time-frame the logs were collected. You’ll need to go back to the “Lets get some logs!” section and try again.
The important parts of this line are the
Possibly instrumenting and the
Possibly instrumentingmeans that the method was called and the .NET CLR presented the method signature to the agent for possible instrumentation.
[assembly]assembly.class.method(parameters)is the exact method signature that must be present in the XML file for the agent to match your xml to the actual method signature. I do mean exact (case matters here).
In that method signature, compare the following:
The portion inside the brackets (
[assembly]in our example) should map to the
assemblyName=attribute in your xml file.
The next portion, up to, but not including the method name (
assembly.Classin our example), should map to the
className=attribute in your xml file.
The next portion, up to, but not including the opening parenthesis (
methodin our example), should map to the
methodName=attribute in your xml file.
The parameters inside the parenthesis are not required in the XML unless you are trying to instrument only certain cases of an overloaded method. In that case, map them to the
parameters=attribute of the
If all of those element match exactly between the method signature in the logs and the one in the xml file, you should then see another line shortly after the
Possibly instrumenting line which is identical with the exception that “Possibly instrumenting” has been replaced with “Instrumenting method”.
[Trace] <<DateTime>> Instrumenting method: (Module: <<path/to/your/module>>, AppDomain: <your domain>>)[assembly]assembly.class.method(parameters)
If you see the “Instrumenting method” line for your method signature, then you can be sure that your custom XML file is being picked up by the agent and applied to your application.
Some common what-if scenarios
SCENARIO 1: I don’t see my method in the profiler log at all! There is no line that says “Possibly instrumenting” and contains my method’s signature.
This means that the .NET CLR never presented the method to the agent for possible instrumentation.
Check the following:
Make sure you’re looking at the right log file. The name of the file should contain the PID of your application’s process at the time the logs were collected.
Make sure the method you’re trying to instrument is actually being called at the time the logs are being generated.
Make sure the logs were generated during an application restart. This all happens at JIT time, so make sure that’s covered.
SCENARIO 2: In the profiler log I get the line that has “Possibly instrumenting” and my method signature, but I don’t get a second line that says “Instrumenting method”.
“Possibly instrumenting” means the .NET CLR saw the method and asked if the agent wanted to do anything with it. “Instrumenting method” means the agent said “yes, I have instrumentation for that method”. So, this scenario means the agent did not find valid instrumentation for that method.
Check the following:
- Your XML is in the right directory (see " Is your XML in the right directory?" above).
- Your XML is valid (see “Is your XML valid?” above).
- The values in your xml match the method signature the CLR reported exactly.
SCENARIO 3: In the profiler log I get both the “Possibly instrumenting” and “Instrumenting method” lines for my method, but I don’t see anything showing up in the New Relic UI.
This situation could be caused by many things that are bigger than this post’s scope (network connectivity, application naming, application duration, etc), but in the context of troubleshooting your custom instrumentation file, check the following:
Make sure you’re not trying to instrument the
main()method (or similar). A transaction consists of everything that happens between the time an instrumented method is called and when it returns. Since the
main()method stays alive for the entire lifecycle of the application, it never returns and nothing is ever reported to New Relic. Instrument things inside
main(), not main itself.
Make sure you’re using (or not using) the
nameattribute correctly in your xml file.
nameis an attribute of the
<tracerFactory>element and tells the agent if it should create a transaction or not when the method is called. If it is set to the value
then it will create a new transaction when the method is called (if one does not exist already). If there’s already a transaction running at the time the method is called, it will add the method call as a segment to the existing transaction.
nameattribute is omitted all together, the method will only be added to a transaction that already exists at the time the method is called. It will not create a new transaction if one does not already exist.
All metrics from the agent must be contained within a transaction. If there’s no transaction, the data won’t get reported to your account.
I hope this has been helpful. Happy monitoring!