Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Relic Solution: Python Agent and the Multiprocessing Library


#1

Not seeing data when using the multiprocessing library and seeing this warning in the logs?

newrelic.core.application WARNING - Attempt to reactivate application or record transactions in a process different to where the agent was already registered for application ‘Multiprocessing’. No data will be reported for this process with pid of 72942. Registration of the agent for this application occurred in process with pid 72911. If no data at all is being reported for your application, then please report this problem to New Relic support for further investigation.

With the Python agent you can either register the agent in the parent process or the child process, but not both. Typically you’ll want to register in the child process. Due to the way multiprocessing terminates the subprocess, you’ll also want to be sure to call shutdown_agent().

One caveat- this will register the agent with every new process which could add some measure of overhead, particularly if there are many, short-lived processes.

Check out the examples below (don’t forget to also import and initialize the agent):

import newrelic.agent
newrelic.agent.initialize('newrelic.ini')

import functools
import multiprocessing

EXAMPLE ONE - Use a decorator if using multiprocessing.Process() directly

def subprocessed_background_task(*bt_args, **bt_kwargs):
    def _register_wrapper(func):
        @functools.wraps(func)
        def _wrap_func(*args, **kwargs):
            newrelic.agent.register_application(timeout=10.0)
            try:
                return newrelic.agent.background_task(*bt_args,
                        **bt_kwargs)(func)(*args, **kwargs)
            finally:
                newrelic.agent.shutdown_agent()
        return _wrap_func
    return _register_wrapper


@subprocessed_background_task()
def say_hello(name):
    print('Hello %s!' % name)


if __name__ == '__main__':
    p = multiprocessing.Process(target=say_hello, args=('New Relic',))
    p.start()
    p.join()

EXAMPLE TWO - Override the run() method when subclassing the process

class NRProcess(multiprocessing.Process):

    def __init__(self, name, *args, **kwargs):
        self.hello_name = name
        super(NRProcess, self).__init__(*args, **kwargs)

    def run(self):
        app = newrelic.agent.register_application(timeout=10.0)

        with newrelic.agent.BackgroundTask(application=app, name='say_hello'):
            print('Hello %s!' % self.hello_name)

        newrelic.agent.shutdown_agent()


if __name__ == '__main__':
    p = NRProcess('New Relic')
    p.start()
    p.join()