Feature Idea: RQ and Introspection on Tasks

Support: Full-Stack Observability (FSO) Agents
,
#1

+1 for this issue. Just migrated from Celery to RQ, and realized introspection on my tasks disappeared.

New Relic edit

  • I want this, too
  • I have more info to share (reply below)
  • I have a solution for this

0 voters

We take feature ideas seriously and our product managers review every one when plotting their roadmaps. However, there is no guarantee this feature will be implemented. This post ensures the idea is put on the table and discussed though. So please vote and share your extra details with our team.

Reporting Django web app + Django-RQ worker
#2

Would you mind sending me information on the workaround please?

#3

Since it’s an experimental workaround (not officially supported), I’ll open a support ticket so we can try to get it working for you and file a feature request for official support of RQ.

You should receive an e-mail shortly with a link to the ticket!

#4

Is there any update on this topic? I’m running RQ on Heroku. Thanks.

#5

@kendallc Thanks for posting to the forum. I’ll be opening a ticket on your behalf a little later outlining the experimental, unsupported workaround we have to instrument RQ.

#6

@hakimu Hi, please could I also have details on this workaround?

#7

@alex_holmes Thanks for your post to the forum. I went ahead and opened a ticket on your behalf and just sent you a response with the experimental workaround to instrument RQ.

If you don’t see the ticket, please check your spam folder and reach out to us if necessary.

#9

For those interested in the experimental workaround (which is not officially supported), I’d like to provide a bit of insight into the issue.

Part of the issue we face when attempting to instrument RQ with the Python agent is that RQ creates a separate process for each task to run in.

The Python agent works by lazily registering when the first transaction occurs. Registration can take up to a couple seconds to complete.

Because of the multiple processes that are created and the time it takes the agent to startup and register, using the Python agent with RQ wouldn’t work well if the tasks are very short and time critical. With the time it takes to register the agent, the minimum reporting period and the time needed to harvest data regarding the task, the worker process can exist for ~5 seconds. Without the Python agent, that time may be sub-second for a short job.

If you’re still interested in the experimental workaround after reading and understanding these caveats, let us know and we can open a support ticket on your behalf.

#10

I am also interested in the workaround. Please let me know! Thanks

#11

@charlieplivo I am going to start a ticket for you. You should receive something in your inbox in a few minutes.

#12

I’ve just started working a project that uses rq but isn’t using this work-around, can you also open a ticket for me? Thanks!

–fran

#13

@fran_kostella I will get a ticket opened for you shortly. Keep an eye on your inbox!

#14

Hi @SamusSam, we face the same issue (details here: Newly defined applications not showing up in Web UI [Heroku/Python]) and would like to investigate the applicability of the workaround for our use case. Can you please send us the details, too? Thanks!

#15

Hi @leonhard.melzer - I’ve opened a support ticket for you. Please watch your email for the the ticket details.

#16

I’m also interested in investigating the workaround. Our tasks take about 5 seconds to complete already, so the slowness may not be a factor for us.

1 Like
#17

Sign me up for this workaround as well. Do we know if this has made it into a roadmap anywhere?

#18
#19
#20