Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

How to instrument celery tasks with newrelic agent?


#1

Hello,
We are using New Relic with a pyramid application, and now we want to monitor our celery workers.

Currently we have Celery 4.1.0 and NR 4.14.0.115, it appears that NR never sends out any reports to collectors.
When the client is ran in debug mode no errors appear.

This is how we normally run our application in preforking mode:

NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program celery worker -c 2 -A app.lib.celery_init --ini=development-template.ini -Q default

The only situation where we got data sent to new relic was when i ran celery using non-prefork mode:

NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program celery worker -c 2 -A app.lib.celery_init --ini=development-template.ini -Q default --pool=solo

But this mode is not suitable for production usage. I’ve also tried setting startup_timeout = 10.0 in our ini file, debug logs did show it configured but it did not seem to help in this scenario.

What else can we try to get NR working with Celery?


#2

Hi @marcin3

It looks like we are working with you on this in an open ticket, but will be sure to update what the solution is in this post once we’ve found out what’s preventing Celery from working properly with the agent.