NewRelic Ruby agent failing to start on Ruby 2.7, Rails 6, with the addressable gem

#1
  • What language agent version are you using? Any other relevant versions? (Gems, etc.)

We’re using Ruby 2.7.0, Rails 6.0.2.2, and the “addressable” gem (v2.7.0, although the same issue happened when we tried downgrading to 2.5.x).

  • Paste in the error message you are seeing (text or screenshot).

During server startup, the NewRelic agent fails to start. Here are some relevant parts of the logs:

#<Thread:0x00007fbcd1b2f878 /.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/newrelic_rpm-6.10.0.364/lib/new_relic/agent/threading/agent_thread.rb:12 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/newrelic_rpm-6.10.0.364/lib/new_relic/agent/threading/agent_thread.rb:14:in `block in create'
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/newrelic_rpm-6.10.0.364/lib/new_relic/agent/agent.rb:718:in `block in start_worker_thread'
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/newrelic_rpm-6.10.0.364/lib/new_relic/agent/agent.rb:691:in `deferred_work!'
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/newrelic_rpm-6.10.0.364/lib/new_relic/agent/agent.rb:672:in `catch_errors'
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/newrelic_rpm-6.10.0.364/lib/new_relic/agent/agent.rb:692:in `block in deferred_work!'
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/newrelic_rpm-6.10.0.364/lib/new_relic/agent.rb:506:in `disable_all_tracing'
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/newrelic_rpm-6.10.0.364/lib/new_relic/agent/agent.rb:693:in `block (2 levels) in deferred_work!'
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/newrelic_rpm-6.10.0.364/lib/new_relic/agent/agent.rb:940:in `connect'
	 ... 10241 levels...
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/activesupport-6.0.2.2/lib/active_support/core_ext/range/each.rb:9:in `each'
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/addressable-2.7.0/lib/addressable/idna/pure.rb:153:in `block in unicode_compose'
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/addressable-2.7.0/lib/addressable/idna/pure.rb:215:in `unicode_compose_pair'
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/addressable-2.7.0/lib/addressable/idna/pure.rb:305:in `lookup_unicode_composition'
/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/addressable-2.7.0/lib/addressable/idna/pure.rb:305:in `hash': stack level too deep (SystemStackError)
  • Describe your issue: what are you seeing? What were you expecting to see?

I’m seeing the agent failing to start. I’m expecting it to start properly.

Here are some links describing something similar to our issue:


Any pointers appreciated!

1 Like
#2

@juraj1,

Hello and thank you for the information you’ve shared.

These errors “stack level too deep” are typically indicative of non-terminating recursion.

In the case of the errors you’re seeing, the Ruby agent is not adding to the stack trace and this is something that you wouldn’t be able to configure. Basically, you’re seeing the VM detecting the infinite loop and stopping it.

Typically we recommend looking deeper into the logs for a url or path file that may point to this recursion in your code.

  • Are you seeing the errors without the Ruby agent?

  • Would you be able to share a permalink to your application? This is only followable by you and our team.

    • In order to create a permalink to any page within the New Relic user interface, scroll to the bottom and click ‘Permalink’ all the way on the right next to ‘Kiosk Mode.’ This will show us the exact page and time period that you are observing.

If this does not get you moving in the right direction let us know and we will collect a list of information to further troubleshoot.

#3

@michaelbrooks
Hi Michael! Sorry for not getting back to you earlier.

Thanks for the reply. The error doesn’t happen without the Ruby agent - we believe it’s specifically because of the combination of “addressable” gem and the Ruby agent.
I don’t have a permalink to share - since we weren’t able to make this work, we don’t have any links to the charts.

Let me know if I can provide more information.

Best,
Juraj

#4

Hello @juraj1

I’m creating a ticket for this issue as we’ll need to gather a bit more information in order to continue troubleshooting.

We would recommend once coming to a solution to share the answer here for other community members to utilize it.

#5

Hey all, I have an update for this thread. We figured out the issue - it was not caused by the addressable gem, but rather http_logger which was using prepend instead of alias chaining on Net::HTTP (and the two don’t work together, as is written in the New Relic docs and elsewhere).

Our fix was to patch the http_logger gem to use alias chaining (https://github.com/wallmine/http_logger). An alternative might be to require it manually, but we didn’t test that.

#6

Thanks for coming back to share that :smiley:

#7

Hello there,

I’ve run into this issue as well. Any updates here?

1 Like
#8

Hi, @tech137: Did you see the solution posted in the thread?

#9

Seriously? Your agent fails to start and the only solution you’re proposing is to use a homebrew fix? :confused:

#10

My project doesn’t use http_logger gem and I’m getting the same error here:

** [NewRelic][2021-01-10 00:49:43 +0000 worker.1 (4)] INFO : Starting Agent shutdown
/app/vendor/bundle/ruby/2.6.0/gems/addressable-2.7.0/lib/addressable/idna/pure.rb:305:in `hash’: stack level too deep (SystemStackError)

#11

Same here, I’m not using http_logger and the app crashes with an error
addressable/idna/pure.rb:153:in block in unicode_compose stack level too deep (SystemStackError)

Ruby 2.7.2 with Rails 6.0.3

#12

We had exactly the same problem, ruby 2.7, rails 6, the exact same error, we are not using http_logger (but probably using some gem that introduces the same error).
I saw in the new relic changelog that in version 6.15.0 (not yet released as of this moment) they had implemented a fix:

’ Default prepend_net_instrumentation to false
Previously, prepend_net_instrumentation defaulted to true. However, many gems are still using monkey patching on Net::HTTP, which causes compatibility issues with using prepend. Defaulting this to false minimizes instances of unexpected compatibilty issues.’

Setting this to false in the new relic config solved our issue :slight_smile:
Hope this helps.

1 Like