Kubernetes Replicaset Desired Pod less than Ready False Positive

I believe I just received a false positive alert k8 cluster alert. Here is the link to the alert if it is visible to the new relic support employees?


Here is a screen shot of the alert condition.

As one can see it is stating the pod desired amount is greater than pods ready for the replica set of did-resolver-5548f7d5d7. However, looking at the cluster everything is nominal. Screenshot below.

Curious why this alert fired given the state of the cluster in screen shots?

Also, more generally, I am curious what such an alert means exactly? It states the replica sets for the a particular pod are not up to the desired amount but replica sets are on a per deployment basis, not pod. Why does the alert condition mention a specific pod?

Thanks for your expertise

Hello, thanks for sharing the screenshots. The Alert Policy in this case refers to a ReplicaSet and not a pod - as you correctly pointed out, the ReplicaSet is not pod-specific.

It looks like the kubectl output does not include resource type replicaset, so it’s possible that this resource (specifically its desired/ready values) was reflecting an unhealthy state at this time - the K8sReplicaSetSample event records for the time period should document that metric and allow investigation.

Our Kubernetes default alert policies are documented, and here is the link for this particular alert: Kubernetes integration: Predefined alert policy | New Relic Documentation

In short, the alert will fire if podsDesired - podsReady is greater than 0 for 5 minutes.