Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

AcquireRequestState is delaying response times - Web API

apm

#1

We’re seeing a lot of our transactions are being delayed in AcquireRequestState for 30+ seconds then the request executes faster. Or it will seemingly timeout with a generic 500 Internal Server Error with no stack trace if it takes about 2 minutes.

I’ve done extensive research on this issue but this event seems to be tied mostly to ASP.Net MVC. We don’t use that (only ASP.Net Web API which does not use session state). We’re at a loss for what could possibly be triggering this event.

We have speculated that perhaps this delay is actually after the event but before the ExecuteRequestHandler event but New Relic reports the AcquireRequestState event for lack of more information.

Has anyone else been experiencing similar issues?


[Resolved] What exactly is System.Web.HttpApplication.ExecuteStep?
#2

Hi @JBurkhart. Here is the typical response we have with this type of issue:

The AcquireRequestState event is raised after the event handler has been created. This IIS pipeline step occurs when ASP.NET acquires the current state (for example, session state) that is associated with the current request.

The first step is to investigate how the application handles state. If there are methods under AcquireRequestState, it will be necessary to implement Custom Instrumentation for the methods that are being called during that pipeline step which are not currently showing up. The assembly name, class name, and method name of the missing methods will need to be determined in order to add them to an instrumentation file. Unfortunately, we are unable provide any more details since we can’t see into the methods that are being called. Look through the application code for anything specifically related to session state, or anything attaching to a specific handler, to see what is going on.

We recommend examining the use of or additions to the AcquireRequestState pipeline step/event handler as well as how the application handles session state in general since this event is where session state is picked up. Somewhere in the code or within the sessions state provider something is being used that is getting hung up. One possibility is a third party library that may have wired up event handlers to the event.

If there are no methods under AcquireRequestState, that indicates that there might be a sessions state issue that is taking up a large amount of time. This may not be occurring all the time, but can occur enough to push the average timing up quite a bit. Examining how the application is handling session state on that particular site to see if there are solutions or workarounds could help. It’s possible there might be an alternative state provider.

The fact that in your case it is a WebAPI application, and does not use session state specifically, does complicate the explanation a bit. However, this is not as much a matter of how the application is coded as it is the way the ASP.NET pipeline itself works. Take a look at the link regarding the IIS pipeline as it explains the process of what is done for every request. Whether the application establishes a specific session state or not, my understanding of the request lifecycle is that the state of the session needs to be determined by the ASP.NET pipeline, regardless if there is some sort of session management occurring or not. Specifically for a WebAPI application, if multiple requests from the same source are being made in a serial context in rapid succession, the entire session state check could be what is causing the bottleneck.

Another question I’d want to ask is whether or not these are outgoing requests, and if so, are they made asynchronously? Changing them to async could very well mitigate the issue. If they’re incoming requests, do you have control of the application making the requests? Again, modifying them to be asynchronous could be one solution.

I’m really interested in anyone else providing some insight on this. Often times we don’t really need to think about how our applications send and receive requests, and what the underlying mechanisms are. We just create the code to do what we need and move on. But sometimes not knowing exactly how IIS, the .NET framework, and our code interact can come back to bite us in the backside. I think this might be one of those situations.

The bottom line is that regardless of the fact that session state is not used in the application, the AcquireRequestState segment is being called. The agent just can’t make that sort of thing up. It really comes to narrowing down where it’s happening, then instrumenting the underlying code to find out what exactly the bottleneck is. Then we can work at finding a solution to the problem.


#3

I’d like to add weight to this issue. We’re seeing AcquireRequestState get called in NewRelic for our API application from time to time, where the duration of the call can be as high as +95,000ms (95 secs). Our API IIS 8.5 application (running on Windows Server 2012 R2) has session state explicitly turned off in its Web.config file (i.e. <sessionState mode="Off" /> ), so it doesn’t make sense that AcquireRequestState would fire and take that long to execute. Any help would be appreciated.


#4

Hi @larry.toppi,

Unfortunately, the issue is not as simple as just turning sessionState off. In fact, one of the key phrases when describing the challenges with AcquireRequestState is the phrase for example, session state when it comes to when this event is raised. In digging deeper into this (actually looking at the .NET source) we can see that this is called when an EventHandler is executed or a RequestNotification object is created. I daresay there are other methods and/or events that, when called, will raise an AcquireRequestState event. Tracking down all of them represents something of a struggle. It seems this is something not talked about a lot outside of the more normalized session state discussions.

The most common place we see this event raised is certainly related to session state management. But there are very obviously outliers where these event can still be raised. It can even be called directly from application code. The thing the agent grapples with is that it can identify the event, but rarely the source. When it is raised as part of the ASP pipeline, the only notification the agent gets is that this is one segment of the transaction. The source, or the methods executed inside of the event, is something the agent is rarely instrumenting by default.

I wish we could offer more insight for you on this. There are a lot of moving parts inside of a .NET application, some of which involve the operating system itself, IIS, the version of .NET, whether or not the methods are asynchronous, application pool settings, execution modes, permissions, etc.

While I don’t want to open a second can of worms here, this harkens to the issue with the lack of stack traces for 500 errors. The agent provides a stack trace when it is offered and/or available. Where the stack trace, if one even exists, occurs within the ASP pipeline is extremely important. Sometimes it occurs before any actual application code is executed. In such cases the error is reported to the application, which in turn lets the .NET agent see and report that an error occurred, but no other details are provided. The agent simply sees that it happened and reports as much information as is possible. Beyond that the agent simply has no further details it can offer.