I’m setting up kubernetes integration in my GKE clusters as described here: https://docs.newrelic.com/docs/integrations/kubernetes-integration/installation/kubernetes-integration-install-configure. I’m using the installer to generate manifest files.
Immediately after applying the manifest it can no longer schedule a new pod: I can’t deploy anything, auto-scaling is not working.
kubectl describe deployment my-deployment output:
Type Status Reason
Progressing True NewReplicaSetCreated
Available False MinimumReplicasUnavailable
ReplicaFailure True FailedCreate
I’m also seeing these errors in the web UI (GKE - Workloads):
- request did not complete within requested timeout
- context deadline exceeded
I narrowed it down to the admission webhook (nri-bundle-nri-metadata-injection) timing out. If I decrease the timeout value to 10 seconds it fixes the issue. But I suspect that the webhook is not actually working: it takes exactly 10 seconds for a new pod to appear. So it’s probably still timing out but
failurePolicy: Ignore saves us.
I tried disabling firewall and network policies - doesn’t help. Also tried in multiple clusters, getting the same results.
How do I check if the webhook is working properly?
Are there some logs worth looking into?