Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Introductory Tornado 4 support in Python Agent 2.62.0.47

tornado
version-python-2-62-0-47
python

#1

Python Agent v2.62.0.47 just came out and includes introductory Tornado 4 support! Find out more about what is covered and the design decisions behind the Tornado 4 work in the release notes and specific documentation. In particular, for this release you need to enable Tornado 4 support in your agent configuration and you’ll want to test it in a non-production environment to make sure it meets your needs.

We’re excited to get this out and very eager for your feedback. Please reply to this thread or contact support with any feedback you have about this first phase of Tornado 4 support.


Python agent does not working with tornado 4
#2

There is a long list of caveats in the release notes. Is NR continuing to work to resolve those missing features, particularly the ones relating to timing and cross application tracing of asynchronous HTTP callouts? Is there an ETA on resolving the caveats listed?


#3

@eric_stein_de Indeed, we are working hard to address the known limitations.

We’re interested in your feedback on what you think the highest priorities should be. For you, would you say that getting cross application tracing working with Tornado’s httpclient belongs at the top of the list?


#4

@toffermann

I’m missing support for tornado.platform.asyncio IOLoop. This is the error I keep getting while using agent version 2.62.0.47 and Tornado 4 app:
2016-04-04 11:53:04,147 (14051/MainThread) newrelic.hooks.framework_tornado_r3.util ERROR - Runtime instrumentation error. Attempting to finalize a transaction which has already been finalized. Please report this issue to New Relic support. File "/project/api/tornado/app.py", line 63, in <module> start_application() File "/project/api/tornado/app.py", line 40, in start_application ioloop.run_forever() File "/usr/lib/python3.4/asyncio/base_events.py", line 244, in run_forever self._run_once() File "/usr/lib/python3.4/asyncio/base_events.py", line 1081, in _run_once handle._run() File "/usr/lib/python3.4/asyncio/events.py", line 120, in _run self._callback(*self._args) File "/project/venv/lib/python3.4/site-packages/newrelic-2.62.0.47/newrelic/hooks/framework_tornado_r3/ioloop.py", line 37, in _nr_wrapper_IOLoop__run_callback_ possibly_finalize_transaction(callback._nr_transaction)


#5

@kamilrs Unfortunately, we don’t yet support the tornado.platform.asyncio IOLoop. We called that out as a known limitation in the release notes, but we do plan to add support in the future. Keep an eye on future release notes for updates.


#6

We’ve found that there is a significant overhead on our application caused by New Relic agent. It comes out, that there is additional 50ms on every request. Is it possible that we do something wrong, or is it a known issue?


#7

I found some error msgs in my app: ( in calling requests.post)

code like this:

import requests
requests.post(url, params, headers)

return request(‘get’, url, params=params, **kwargs)
File “/usr/local/lib/python3.5/site-packages/newrelic-2.62.0.47/newrelic/api/external_trace.py”, line 124, in dynamic_wrapper
return wrapped(*args, **kwargs)
File “/usr/local/lib/python3.5/site-packages/requests/api.py”, line 50, in request
response = session.request(method=method, url=url, **kwargs)
File “/usr/local/lib/python3.5/site-packages/requests/sessions.py”, line 465, in request
resp = self.send(prep, **send_kwargs)
File “/usr/local/lib/python3.5/site-packages/newrelic-2.62.0.47/newrelic/api/external_trace.py”, line 124, in dynamic_wrapper
return wrapped(*args, **kwargs)
File “/usr/local/lib/python3.5/site-p ackages/requests/sessions.py”, line 573, in send
r = adapter.send(request, **kwargs)
File “/usr/local/lib/python3.5/site-packages/requests/adapters.py”, line 370, in send
timeout=timeout
File “/usr/local/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py”, line 544, in urlopen body=body, headers=headers)
File “/usr/local/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py”, line 349, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr


#8

@dongyi and @glasek

To get the most detail possible, I have opened support tickets for you both. Once we solve your issues, we will post a summary to this forum thread.

Please check your emails for the support ticket.


#9

After doing some additional investigation into the issues that @dongyi and @glasek reported, we have filed two new issues which we plan to fix in the next release of the agent.

@dongyi was getting an AttributeError: 'NoneType' object has no attribute 'cross_application_tracer' error thrown because the application hasn’t registered by the time a particular external request is being made, so there’s no settings object attached to the transaction yet, but the external trace instrumentation expects a valid settings object. They only see this error when they first start the application, but not afterwards. Once the agent has registered, the settings object is valid, so there’s no attribute error.

@glasek was seeing some additional response time due to an issue with our instrumentation for tornado.httpclient. If an application made outgoing HTTP requests to many different URLs, the agent was creating and storing many more metrics than normal for each transaction, which we believe was contributing to the increased response times that they are seeing.

Both of these issues are now being tracked by our Python Agent Engineers and will be corrected in the next agent release. Thanks to both @dongyi and @glasek for helping us improve our Tornado 4 support!


#10

We’re looking forward to getting notified about the next version. Thanks for your support :slight_smile:


#11

Yes, cross application tracing and being able to view/trace multiple asynchronous concurrent outbound callouts and see their timings - that’s key.


#12

Thanks for adding Tornado 4 support. I plugged it into our app and had a couple issues. Where can I post details and tracebacks? Thanks!


#13

@steve6 Hi there, we would be happy to open an internal support ticket for you and better collect traces and discuss the issues you are seeing. After we get your issue resolved, I will post a summary back to the forum for future readers benefit.

Check your email for a message from me!


#14

We have just released a new version of the Python agent that contains improvements to our Tornado 4 support!

The issues that @dongyi and @glasek reported have been addressed in this release along with other improvements. Thanks again for your reports and suggestions.

Please review the Release Notes for version 2.64.0.48 to see all the changes.


#15

@agoetz

We just tested new release with @glasek and we can say that we see a big improvement from previous release. The overhead time is now acceptable (now it is ~10ms previously it was ~100ms). Number of errors in log is now also much lower than previously. Once per 3-4 seconds we get this error

Apr 21 12:58:34 app04 newrelic-admin[18613]: ERROR:newrelic.hooks.framework_tornado_r3.util:Runtime instrumentation error. Attempting to finalize a transaction which has already been finalized. Please report this issue to New Relic support. Apr 21 12:58:34 app04 newrelic-admin[18613]: File "/home/webadmin/app-app/current/api/tornado/app.py", line 58, in <module> Apr 21 12:58:34 app04 newrelic-admin[18613]: start_application() Apr 21 12:58:34 app04 newrelic-admin[18613]: File "/home/webadmin/app-app/current/api/tornado/app.py", line 34, in start_application Apr 21 12:58:34 app04 newrelic-admin[18613]: tornado.ioloop.IOLoop.current().start() Apr 21 12:58:34 app04 newrelic-admin[18613]: File "/home/webadmin/app-app/current/venv/lib/python3.4/site-packages/tornado/ioloop.py", line 827, in start Apr 21 12:58:34 app04 newrelic-admin[18613]: self._run_callback(callback) Apr 21 12:58:34 app04 newrelic-admin[18613]: File "/home/webadmin/app-app/current/venv/lib/python3.4/site-packages/newrelic-2.64.0.48/newrelic/hooks/framework_tornado_r3/ioloop.py", line 37, in _nr_wrapper_IOLoop__run_callback_ Apr 21 12:58:34 app04 newrelic-admin[18613]: possibly_finalize_transaction(callback._nr_transaction)

And yes we are using heavily tornado async http client which might be the cause.


#16

@agoetz

I found one more thing which is stopping us from using NR with tornado on production. As I mentioned earlier, we make a lot of requests outside of our system. If a request from outside returns error code 4XX or 5XX, NR it treats it as error in our application even when it is handled. This makes NR raise alerts when our application is working properly.


#17

Thanks for upgrading and trying the newest agent @kamilrs!

I know the feedback you have provided will be useful for our development team as they work to identify the cause of and implement improvements for these issues. Stay tuned for future updates and possibly a follow-up question or two from us.


#18

Hey @kamilrs you can ignore those errors in your config file

https://docs.newrelic.com/docs/agents/python-agent/installation-configuration/python-agent-configuration#error-collector-settings

Hope this helps


#19

I have a issue, when I run tornado in docker I can’t get metric, but tornado on host and flask in docker still work. Newrelic doesn’t support tornado in docker, does it ?


#20

@thanhdl Thanks for posting to the forum. There shouldn’t be an issue running the Python agent with Docker. Do you have the agent installed within your Docker container? If you do and you aren’t seeing data, please let us know. We’ll gather some additional info to diagnose your issue.