Sidecar agent with Apache integration on AWS Fargate - am I doing this wrong?

I have an Apache server running on AWS ECS/Fargate, and have already setup the newrelic_infra image as a sidecar that gives me metadata on the ECS cluster, i.e. task counts, container CPU and memory usage. This works and shows up in NR.

I also want process metrics from mod_status, similar to what you get with the on-host integration (apache children idle/busy count, per-process mem/cpu, etc)

I cleverly (I thought) built a custom docker image based on newrelic/infrastructure image. It downloads and installs the nri-apache_linux integration. I deployed this custom image as the sidecar in my task definition, and I can see it is happily requesting and getting the status url from Apache as it should (GET /server-status?auto every 1 minute)

Howeverrrrrr… nothing is showing up in NR. I feel this must be due to the fargate/ECS switches being thrown in the environment for the container (got these from the documentation). I’m not sure how the agent’s internals work.

The ECS metrics are still working fine, so I know the license key is OK. Just not seeing anything under “on host” or anywhere in Data explorer.

Does anyone know if there is a better way to go about this? If this is a sensible way, any reason this data from the apache integration wouldn’t be making it to NR?

Details below:

Here’s how I know the integration in the sidecar is doing its thing:

Apache container logs have stuff like this once a minute: 127.0.0.1 - - [11/Aug/2021:19:21:33 +0000] “GET /server-status?auto HTTP/1.1” 200 646 “-” “Go-http-client/1.1” 0
and from a shell in the newrelic_infra container I can do this
bash-5.1# ./bin/nri-apache --metrics
{“name”:“com.newrelic.apache”,“protocol_version”:“3”,“integration_version”:“1.6.1”,"data [{“metrics”:[{“event_type”:“ApacheSample”,“net.bytesPerSecond”:0,“net.requestsPerSecond”:0,“port”:“80”,“server.busyWorkers”:1,“server.idleWorkers”:11,“server.scoreboard.closingWorkers”:0,“server.scoreboard.dnsLookupWorkers”:0,“server.scoreboard.finishingWorkers”:0,“server.scoreboard.idleCleanupWorkers”:0,“server.scoreboard.keepAliveWorkers”:0,“server.scoreboard.loggingWorkers”:0,“server.scoreboard.readingWorkers”:0,“server.scoreboard.startingWorkers”:0,“server.scoreboard.totalWorkers”:12,“server.scoreboard.writingWorkers”:1,“software.version”:“Apache/2.4.29 (Ubuntu)”}],“inventory”:{},“events”:[]}]}

Here’s the container definition for the sidecar (the apache portion elided):

  {
"environment": [
  {
    "name": "NRIA_OVERRIDE_HOST_ROOT",
    "value": ""
  },
  {
    "name": "NRIA_IS_FORWARD_ONLY",
    "value": "true"
  },
  {
    "name": "FARGATE",
    "value": "true"
  },
  {
    "name": "ENABLE_NRI_ECS",
    "value": "true"
  },
  {
    "name": "NRIA_PASSTHROUGH_ENVIRONMENT",
    "value": "ECS_CONTAINER_METADATA_URI,ENABLE_NRI_ECS,FARGATE"
  },
  {
    "name": "NRIA_CUSTOM_ATTRIBUTES",
    "value": "{\"nrDeployMethod\":\"downloadPage\"}"
  }
],
"secrets": [
  {
    "valueFrom": "/newrelic-infra/ecs/license-key",
    "name": "NRIA_LICENSE_KEY"
  }
],
"cpu": 256,
"memoryReservation": 256,
"image": "${newrelic_sidecar_image}",
"name": "newrelic-sidecar",
"essential": false,
	"logConfiguration": {
		"logDriver": "awslogs",
		"options": {
			"awslogs-group": "/fargate/service/${app_name}-newrelic-infra-${app_environment}",
			"awslogs-region": "${aws_region}",
			"awslogs-stream-prefix": "ecs"
		}
	},
"networkMode": "awsvpc"

}

The dockerfile for my custom image looks like this

FROM newrelic/infrastructure:latest

# downloads the nri-apache integration and unpacks to /, puts the binary in right place
ADD https://download.newrelic.com/infrastructure_agent/binaries/linux/amd64/nri-apache_linux_1.6.1_amd64.tar.gz /
RUN tar -xzf nri-apache_linux_1.6.1_amd64.tar.gz && rm nri-apache_linux_1.6.1_amd64.tar.gz

# copy our sample sample config to the real config
COPY apache-config.yml /etc/newrelic-infra/integrations.d/apache-config.yml

And the apache-config.yml has this:

integration_name: com.newrelic.apache

instances:
    - name: apache-server-metrics
      command: metrics
      arguments:
          status_url: http://127.0.0.1/server-status?auto
          remote_monitoring: true
      labels:
          env: production
          role: load_balancer

@sbeam As per our documentation we support a small subset of the network types available to the ECS cluster nodes. Those types are the none and bridged network modes. As per the aws documentation the only network mode that can be configured for the Fargate launch type tasks is “awsvpc” which falls under the unsupported network types. Meaning that monitoring services on ecs fargate is currently not supported.

If you would like to file a feature request you should be able to do that through the Feature Idea area in the discussion forum.

Interesting. Supported or not, I was able to get it to work with the custom image, and the data actually was being shipped to NR under the ApacheSample namespace. Queries and charts are working as expected.

1 Like