We are moving away from using the
container_memory_usage_bytes metric in favor of
container_memory_working_set_bytes for memory monitoring within our Kubernetes capabilities.
container_memory_usage_bytes metric isn’t an accurate indicator for out of memory (OOM) prevention as it includes cached data (i.e., filesystem) that can evict in memory pressure scenarios. However, the
container_memory_working_set_bytes metric excludes cached data and is what Kubernetes uses for OOM/scheduling decisions, making it a better metric for monitoring/alerting memory saturation.
What is changing?
This change will be reflected within New Relic One as follows:
Kubernetes integration: Predefined alert policy. The “Container memory usage % is too high” alert condition now uses the new
memoryWorkingSetUtilizationattribute instead of
(memoryUsedBytes/memoryLimitBytes)/100. Please note that existing alert conditions won’t be automatically updated. If you are using our predefined alert policy, we strongly recommend you to update it accordingly.
The following widgets are now using working set bytes in Kubernetes dashboard:
- Resources Used
- Container Memory Usage - % Used vs Limit
- Container MBytes of Memory Used
- Node Resource Consumption
Golden metrics definition for containers will now use
The Kubernetes cluster explorer now shows the container’s memory consumption compared to the limit, using the
WorkingSetBytesmetric in the Pod preview
New alerts can now be created based on:
- Memory Working Set Bytes
- Memory Working Set Utilization
- Requested Memory Working Set Utilization