I am unable to forward EKS Fargate logs to New Relic

Support: Telemetry Data Platform (TDP) Log Management
, ,
#1

I am using EKS with fargate, I want to ship logs from fargate to the new relic. I have installed new relic agent through helm and Terraform. The values.yaml is below.

global:
  licenseKey: <>
  cluster: <>
  fargate: true  
prometheus:
  enabled: false
  lowDataMode: true 
webhook:
  enabled: true
kubeEvents:
  enabled: true
logging:
  enabled: true
ksm:
  enabled: true
newrelic-infra-operator:
  enabled: true

Logs of containers running on EC2 are present in newrelic but not of fargate. EKS has both EC2 instance and fargate. Also, Fargate profiles does not contains namespace of newrelic

#2

Hello engineering56, thanks for writing to us and we hope you’re having a great day. First thing we may want to looks at: Is the infra agent sidecar within the fargate node installed and working properly? Please fargate doc page here:
https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/installation/install-fargate-integration/
This page details the install of the fargate integration so we can verify everything was installed correctly. If the install is correct there are a few different options for log forwarding.
If you are using Firelens plugin or CloudWatch for your EC2 logs, both those methods will also work for Fargate logs:

Our Firelens Integration doc page:
https://docs.newrelic.com/docs/logs/forward-logs/aws-firelens-plugin-log-forwarding

I recommend using Firelens and our integration to forward Fargate logs to New Relic.

#4

Is Firelense doc for ECS? I am currently using EKS with fargate.

Sidecar is running fine. Even I can see Fargates as a Node on Newrelic Dashboard

#5

Hi engineering56, yes that doc above is for ECS. We do not have docs specific to this situation yet as EKS Fargate support is still in beta and the second yellow box on our Fargate page shows logging is not yet supported.

#6

Hey! Just getting started on EKS Fargate and was sad to see it does not support logging to New Relic. Currently looking into sending logs to CloudWatch first and then pipe CloudWatch logs to New Relic. Is the New Relic team actively working on adding the EKS Fargate Logging? Is this something we can expect in the near future? Thanks!

#7

Hi @hadrien2

Thanks for reaching out, if your enquiring about sending logs to CloudWatch it may be worth revising the below docs.

  1. Introduction to the Amazon CloudWatch Metric Streams integration | New Relic Documentation

  2. AWS Lambda for sending CloudWatch logs | New Relic Documentation

I hope the above was helpful, please do reach out with any questions you may have.