[Lambda] New Relic Lambda Extension not working without CloudWatch

Support: Full-Stack Observability (FSO) Serverless
, , ,
#1

Greetings.

I’ve been trying to make NewRelic-lamba CLI to instrument my functions as well as sent my logs straight from Lambda to NewRelic.

Back in December 2021 I nailed the process and made it work using the instructions listed on the newrelic-lambda cli Repo:

I made sure my AWS account and NewRelic accounts were connected, instrumented the functions, subscribed to the stream with no filter (as this makes sure I can send everything)
and had it all working.

A few weeks later - in January, I attempted the same process again and failed. Then I decided to run the functions I had configured in December, and they also failed.

Now I’m stuck. I went over the steps a dozen times and the instrumentation work - I get to see the invocations and even have access to the NewRelic logs, but once I start skiping cloudwatch to send logs into NewRelic: I got nothing.

Is there any information I can provide to help with this?

I already tried every method I could find, including unlinking and linking the accounts again. I currently have two different AWS accounts linked to NewRelic and none of them can produce results other than coping it from Cloudwatch.

I’m currently testing a python 3.8 function using the proper layer for the region in question and all the environment variables set on both the function and ‘newrelic-log-ingestion’

#2

Hey @tech.monitor,

Thanks for posting the issue you’re facing with the Extension setup! One thing you can do is enable debug logging for the Extension to see if any errors are happening.

NEW_RELIC_EXTENSION_LOG_LEVEL: DEBUG

Make sure to delete your CloudWatch subscription filter to avoid sending invocation telemetry to New Relic twice: one through CloudWatch and one through the Extension.

newrelic-lambda subscriptions uninstall \
    --function FUNCTION_NAME \
    --aws-region YOUR_REGION

The other piece that can cause failures for the Extension is the AWS Secrets Manager. We store your license key in the Secrets Manager and thus need an add-on policy to allow the function to:

secretsmanager:GetSecretValue

https://discuss.newrelic.com/t/lambda-troubleshooting-framework-general-knowledge-part-1/146356#heading–secrets-manager-role

It is possible to bypass the Secrets Manager by adding a license key environment variable to your function.

NEW_RELIC_LICENSE_KEY: YOUR_LICENSE_KEY_NOT_API_KEY

Some additional environment variables you can set for use with the Extension.

If you can share any details from CloudWatch logs that include the [NR_EXT] lines, that would be helpful. Best of luck!

#3

Hello,
Just curious if you had any luck with the Extension and having it push logs into New Relic?

I recently started using the Lambda for log ingestion, mostly because I didn’t realize the Lambda extension could do this. I don’t know which way is preferred/better, but this thread makes me think I should stick with the older subscription + Lambda method for now.

Thanks,
Jason

#4

Hello all! Tagging in @tech.monitor to see if they have any feedback for you @JCapriotti - I’m not sure if this resolved the issue for them or not.

#5

I was just looking this over again… Is there a full set of documentation for how to configure the Lambda Extension, particularly for Cloudwatch logging?

#6

Hello @JCapriotti -

This documentation covers all the configuration you can do on the Lambda Monitoring Integration:
https://docs.newrelic.com/docs/infrastructure/infrastructure-integrations/cloud-integrations/configure-polling-frequency-data-collection-cloud-integrations/

Let me know if that does not get you what you are looking for.

1 Like