Feature Idea: Support for Python 3 Asyncio ioloop ETA

For those of you using the asycio-based aiohttp server, I have more good news. Yesterday we released version 2.96.0.80 of the Python agent with out of the box instrumentation support for aiohttp server.

Upgrade to the latest agent and let us know what you think!

@jstuckey that’s great! thanks for all of this :).

Something I would like to ask is if you have plans to make function traces to work with asyncio? I’ve been recording transaction metrics for +1w, but function traces haven’t been reported (or are not shown in the ui).

Anyway, I started to move some of the asyncio-based processes that I own from usingn aiometrics to newrelic, thanks for making this happen!

Hi @dmo !

Newer versions of the Python Agent should support Function Traces with coroutines. Here’s some example code showing how to decorate coroutines so that they’re function traced!

import newrelic.agent
import asyncio

@newrelic.agent.function_wrapper
def coroutine_function_trace(wrapped, instance, args, kwargs):
    transaction = newrelic.agent.current_transaction()

    coro = wrapped(*args, **kwargs)

    if not transaction:
        return coro

    name = newrelic.agent.callable_name(wrapped)

    @asyncio.coroutine
    def _tracer():
        with newrelic.agent.FunctionTrace(transaction, name):
            result = yield from coro
        return result

    return _tracer()

@coroutine_function_trace
async def my_coroutine():
   await asyncio.sleep(0.5)

awesome @afeldman, thanks. I was using a newrelic.agent.function_trace decorator for the coroutine and didn’t work, will try this way

Hey @dmo! It’s been a few days—did @afeldman’s proposed solution work for you? Keep us posted—I know the rest of the community will want to hear a follow up from you!

Hey @Linds, I’m eager to try this, unfortunately I have not been able to do that yet.

I’ll share feedback once I tested it

Working code (python 3.5+):

import os

import newrelic.agent

newrelic_app_register_timeout = int(os.environ.get('NEW_RELIC_APP_REGISTER_TIMEOUT', 10))
newrelic_app = newrelic.agent.register_application(timeout=newrelic_app_register_timeout)


def async_newrelic_background_task(f):
    async def wrapper(*args, **kwargs):
        transaction = newrelic.agent.current_transaction()
        if transaction:
            return await f(*args, **kwargs)

        name = newrelic.agent.callable_name(f)
        with newrelic.agent.BackgroundTask(newrelic_app, name):
            return await f(*args, **kwargs)

    return wrapper

How to use:

from my_module import async_newrelic_background_task


@async_newrelic_background_task
async def my_coroutine():
    await sleep(0.5)
    return 'my_coroutine'

Thanks for sharing this great example @allisson!

Hi @dmo and @allisson !

New Relic has released support for tracing Python coroutines using the existing time trace decorator APIs (such as function_trace)!

Details are in the release notes! Any feedback would be appreciated!

Here’s a working example using function trace on a coroutine

import newrelic.agent
newrelic.agent.register_application(timeout=10.0)

import asyncio

@newrelic.agent.function_trace()
async def my_coroutine():
    await asyncio.sleep(0.5)


@newrelic.agent.background_task(name='task')
def main():
    loop = asyncio.get_event_loop()
    loop.run_until_complete(my_coroutine())


if __name__ == '__main__':
    main()

Hi,

This only works if you call asyncio once. I was trying with an infinite loop (our usual case) and I can only see calls count without any details or execution time (it always below 0.1 ms even if there is long sleep inside).

Example code:

import asyncio

import newrelic.agent
newrelic.agent.register_application(timeout=10.0)


@newrelic.agent.function_trace()
async def my_coroutine():
    await asyncio.sleep(0.5)


@newrelic.agent.background_task(name='task')
async def some_task():
    await my_coroutine()


async def main():
    while True:
        await some_task()


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

Hi @grzegorz.kocjan !

Unfortunately the background_task decorator cannot currently be used for coroutines.

For your use-case above, I’ve gone through and reorganized it slightly to fit within those limitations. The following code should work:

import asyncio

import newrelic.agent
application = newrelic.agent.register_application(timeout=10.0)


@newrelic.agent.function_trace()
async def my_coroutine():
    await asyncio.sleep(0.5)


async def some_task():
    await my_coroutine()


async def amain():
    await some_task()


def main():
    loop = asyncio.get_event_loop()
    while True:
        with newrelic.agent.BackgroundTask(application=application, name='task'):
            loop.run_until_complete(amain())


if __name__ == '__main__':
    main()