Error: You're not collecting metric data (even though k8 is integrated in New Relic)

Hi,
Here is our setup

  • kubernetes cluster in AWS cloud
    - Installed kubernetes integration in NewRelic.
    Following pods are running:

newrelic-bundle-kube-state-metrics-744dfb4584-l4t2q 1/1 Running 0 14d
newrelic-bundle-newrelic-infrastructure-2scdf 1/1 Running 0 14d
newrelic-bundle-newrelic-infrastructure-8j7kj 1/1 Running 2 14d
newrelic-bundle-newrelic-infrastructure-dxm4m 1/1 Running 0 14d
newrelic-bundle-newrelic-infrastructure-njfzv 1/1 Running 0 14d
newrelic-bundle-newrelic-infrastructure-pgzg9 1/1 Running 1 14d
newrelic-bundle-newrelic-infrastructure-rnptk 1/1 Running 0 14d
newrelic-bundle-newrelic-logging-brsbf 1/1 Running 0 14d
newrelic-bundle-newrelic-logging-jfbwm 1/1 Running 0 14d
newrelic-bundle-newrelic-logging-pg8vd 1/1 Running 0 14d
newrelic-bundle-newrelic-logging-rlkgk 1/1 Running 0 14d
newrelic-bundle-newrelic-logging-v2sf7 1/1 Running 0 14d
newrelic-bundle-newrelic-logging-xttxt 1/1 Running 0 14d
newrelic-bundle-nri-kube-events-7d55b67b7c-5wtk5 2/2 Running 0 14d
newrelic-bundle-nri-metadata-injection-866765b79-gmndq 1/1 Running 0 14d

  • We also installed kubernetes metric server in kube-system namespace.

Issue:

When trying to get “Meteric Explolrer” for any environment under Infrasstructer->Kubernetes" getting following error

image

In newrelic-infrastructure pod , has following error:

time=“2021-02-05T13:15:56Z” level=error msg=“metric sender can’t process” component=MetricsIngestSender error=“error sending events: Post “https://infra-api.newrelic.com/infra/v2/metrics/events/bulk”: read tcp 10.147.40.244:52950->162.247.243.146:443: read: connection reset by peer” postCount=66376 sendErrorCount=1
time=“2021-02-06T12:31:24Z” level=info msg=“Removing inventory cache” agentEntityIDChanged=false component=PatchSender entityKey=“k8s:cluster:ucm-ng-dev-cluster” offlineTime=24h0m0s

Hi @cchokshi,

Thanks for your question and for providing the log message, that was very helpful!

The errors you have reported typically indicate an issue with the Infrastructure agent’s communication being blocked by the network from reaching New Relic endpoints. If the agent encounters these errors, it simply caches the data and tries again up to a certain point.

In order to report data to New Relic, our infrastructure monitoring needs outbound access to a list of domains, networks, and ports which can be found in our documentation here: