Enabling Python Agent causes crash in inspect.getfullargspec for builtin

We have a Flask (1.1.1) server with Flask-SocketIO (4.2.1) running in a Python 3.6.11 docker container. It is started directly, via a main.py:
python3 -u main.py. It works great. When I change the command line to NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program python3 -u main.py, with the settings file having monitor_mode = false, it still works great. When I enable monitor_mode, it crashes at startup while initializing the injection kernel, which internally is using the inspect.getfullargspec function. The mysterious part is that it is failing for a built-in method.

The full error is below:

Traceback (most recent call last):
galileo-server_1         |   File "/usr/local/lib/python3.6/inspect.py", line 1126, in getfullargspec
galileo-server_1         |     sigcls=Signature)
galileo-server_1         |   File "/usr/local/lib/python3.6/inspect.py", line 2273, in _signature_from_callable
galileo-server_1         |     skip_bound_arg=skip_bound_arg)
galileo-server_1         |   File "/usr/local/lib/python3.6/inspect.py", line 2097, in _signature_from_builtin
galileo-server_1         |     raise ValueError("no signature found for builtin {!r}".format(func))
galileo-server_1         | ValueError: no signature found for builtin <FunctionWrapper at 0x7f0e91684388 for functools.partial at 0x7f0e916e4408>
galileo-server_1         |
galileo-server_1         | The above exception was the direct cause of the following exception: 
galileo-server_1         | 
galileo-server_1         | Traceback (most recent call last):
galileo-server_1         |   File "main.py", line 7, in <module>
galileo-server_1         |     app = create_app()
galileo-server_1         |   File "/app/app/__init__.py", line 208, in create_app
galileo-server_1         |     _configure_dependency_injection(app, None)
galileo-server_1         |   File "/app/app/__init__.py", line 104, in _configure_dependency_injection
galileo-server_1         |     modules=modules,
galileo-server_1         |   File "/usr/local/lib/python3.6/site-packages/flask_injector.py", line 317, in __init__
galileo-server_1         |     process_dict(container, injector)
galileo-server_1         |   File "/usr/local/lib/python3.6/site-packages/flask_injector.py", line 344, in process_dict
galileo-server_1         |     d[key] = wrap_fun(value, injector)
galileo-server_1         |   File "/usr/local/lib/python3.6/site-packages/flask_injector.py", line 78, in wrap_fun
galileo-server_1         |     return wrap_fun(inject(fun), injector)
galileo-server_1         |   File "/usr/local/lib/python3.6/site-packages/injector/__init__.py", line 1354, in inject
galileo-server_1         |     bindings = _infer_injected_bindings(function, only_explicit_bindings=False)
galileo-server_1         |   File "/usr/local/lib/python3.6/site-packages/injector/__init__.py", line 1159, in _infer_injected_bindings
galileo-server_1         |     spec = inspect.getfullargspec(callable)
galileo-server_1         |   File "/usr/local/lib/python3.6/inspect.py", line 1132, in getfullargspec
galileo-server_1         |     raise TypeError('unsupported callable') from ex
galileo-server_1         | TypeError: unsupported callable

The only difference is enabling monitor_mode, which of course is the point. Is there something that the newrelic agent is doing to the execution environment that I should be aware of?

Hiya @charlie15!

I’m unable to replicate this using the following (mind ya, it’s a super simple script):

I’ll follow up on this with the engineers for their insight. But please let me know your thoughts on the above.

Cheers!

I will have a look at what you came up with. After looking at this for a few days, I suspect that the problem might be in multiple monkey patching of the core libraries. We are using Eventlet, which patches many core libraries to enable cooperative yielding for multitasking, and I suspect it may have a part to play. I’ll mess with your script and try and post a recreation.

Thanks!

1 Like

Sounds good! Also, what version of the agent is installed?

We’re using the latest version, looks like it’s not locked in our requirements.txt.

I cloned your repo and have been working with it for a while, and all of my expected recreations have nothing to do with NewRelic

`import inspect
import functools

inspect.getfullargspec(functools.partial)`

Will crash every time, no matter how much newrelic is involved. What I can’t figure out is why enabling newrelic in my code changes the code such that it ever makes this 100% deadly call. I will continue working with the recreate, if only for my own purposes, but at this point in time I’m pretty sure it’s not you, it’s me.

Thanks for taking the time to look at this!

1 Like

Hello Charlie,

Just checking in on how you are doing, and do let us know how it goes!

Cheers!