Serverless monitoring: Lambda max memory has no data

I am using Serverless monitoring for AWS Lambda (with the NR lambda extension). All lambda metrics are populated successfully on dashboards, except for “max memory used”. I can’t find any documentation or support topics about this, any assistance would be welcome.

Hey @matthew.bell,

You might need to specify a longer time period. If no data has arrived since 12 hours ago, the chart will be empty. For example:

SELECT average(cwMaxMemoryUsed) FROM AwsLambdaInvocation SINCE 7 DAYS AGO

The cwMaxMemoryUsed attribute is defined in a closed-source repository. There isn’t any public documentation for this attribute as of yet. When available, it will appear in our data dictionary.
https://docs.newrelic.com/attribute-dictionary/?event=AwsLambdaInvocation

I’ll make sure to bring this up with our Lambda Engineering Team in the next meeting. Thanks for bringing this omission to our attention!

I see this issue on all of our instrumented Lambdas, for any time period.

I thought this would be a widely used attribute because it is on the default Lambda dashboard in New Relic One, but as you mention it doesn’t seem to be in the AwsLambdaInvocation event attribute list

How can I troubleshoot why this is not appearing?

@matthew.bell,

This may be an issue with the Metric Streams integration. Our engineering team is currently looking into a similar issue. I will make sure they hear about this as well. Thanks for writing about this!

Note, I am seeing two instrumented functions that have data for cwMaxMemoryUsed.
https://one.newrelic.com/-/09MR2EOYkwY

I’m not sure why the other instrumented functions are not showing anything.

I spoke with the engineering team responsible for the Metric Streams integration and they don’t think it is an issue there. The cwMaxMemoryUsed attribute is maintained by the Lambda monitoring team. Looking into it there.

The only difference I can see is that the functions that are missing data for this attribute are using the aws-chalice/1.23.0 runtime. Functions that appear to be working are either Node.js functions or using the generic python3.8 runtime.

cwMaxMemoryUsed is pulled from the Report log line in CloudWatch. Engineering recommends to check CW logs to make sure that report line exists. It would be interesting to see the aws-chalice json file and CW logs from an invocation that is missing the max memory value. If Chalice is somehow overriding the log formatting, we might not be picking it up.

This would be easier to do in support ticket, so I’ll go ahead and create one to dig into this deeper.

I am experiencing the same issue. My AWS lambda is NodeJS based and uses the following NR layer for sending telemetry data

Hey @subscriptions1,

Thanks for posting about this! That looks like our latest layer for NodeJS14. Can you confirm the function is showing as instrumented?

  1. Is the cwMaxMemoryUsed missing on every invocation?
  2. Are you using the Extension (built into our layer) or aws-log-ingestion function to send telemetry?
1 Like