Your data. Anywhere you go.

New Relic for iOS or Android


Download on the App Store    Android App on Google play


New Relic Insights App for iOS


Download on the App Store


Learn more

Close icon

Synthetic minion for kubernetes

synthetics
kubernetes

#1

We are looking to install a private containerized minions in our clusters per the documentation ~> https://docs.newrelic.com/docs/install-containerized-private-minions

I had a few questions/concerns.

This looks to be for docker install on hosts .

(I) Is this adaptable for Kubernetes ?
(ii) If it is , is there any reference documentation or can you provide any guidelines on how to do this specifically for kubernetes?
(iii ) if we are to deploy it for kubernetes , would it be as a daemonset or as a deployment with an x amount of replicas ( what would be an ideal number ) ?

Thank you and looking forward to your response.


#2

Hey @KOfoegbu Currently the Containerised Private Minion is in a closed Beta phase, so it is being supported by the Development team. I’ll get them looped in here.

I noticed that your Account ID is not listed on the closed beta, and as far as I know there is no way for new accounts to be added now. That said if you’d like updates on a public beta, you can reach out to your New Relic Account Executive.


#3

Ok thanks @RyanVeitch , we are trying to get it set up in our kubernetes cluster and any additional information that the development team can provide would be appreciated.


#4

Hey @KOfoegbu - I’ve heard back from our development team with the information below;

We don’t have any documentation on Kubernetes deployments yet, that said we didn’t have to make any modifications to the minion to deploy it to a local cluster.

Below is our example deployment, but you may be looking a deployment with some number of replicas, where the number of replicas will depend on the types of jobs their running and how many of said jobs are running. You can also take a look at our monitoring documentation that’s linked in our Containerised Private Minion install docs to help you figure out how many replicas you would want.

We are seeing some requests to make the Containerised Private Minion more Kubernetes friendly, so we’ll get your request for that added too.

Please find below our sample Kubernetes deployment.

Kubernetes Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: containerized-private-minion
spec:
  selector:
    matchLabels:
      app: containerized-private-minion
  replicas: 3 # tells deployment to run 3 pods matching the template
  template:
    metadata:
      labels:
        app: containerized-private-minion
    spec:
      containers:
      - name: containerized-private-minion
        image: quay.io/newrelic/synthetics-minion:latest
        volumeMounts: 
          - mountPath: /var/run 
            name: docker-sock
          - mountPath: /tmp
            name: tmp
        env:
          - name: MINION_PRIVATE_LOCATION_KEY
            value: <your key>
          - name: MINION_CHECK_TIMEOUT
            value: "300"
        ports:
        - containerPort: 80
      volumes: 
        - name: docker-sock 
          hostPath: 
            path: /var/run
        - name: tmp
          hostPath:
            path: /tmp

#5

Thanks @RyanVeitch for the speedy response. I will go ahead and test it out.


#6

Great! Let us know how it goes @KOfoegbu :smiley:


#7

Hi @RyanVeitch thanks for the manifest file example. I was able to wrap a helm chart around it and it works just fine. However, I had a question.

We used the latest tag for the image , but it shows this message in the console

" There’s a new minion version out! Upgrade Now"

Just needed to get some clarification on this.

Thanks.


#8

Hey @KOfoegbu - I’m not completely sure why the console says that. My thoughts right now are that the console is looking for the image of the latest private minion VM, instead of the containerised private minion (CPM).

Since the CPM is still in Beta, I expect that the console UI may not be updated to look for it, and still looks at the older VM private minion.

I’ll reach out to the development team for confirmation or clarification on this. :smiley:


#9

Ok , thanks , I will wait for the response from the development team.

Thanks for your speedy response @RyanVeitch


#10

Hey @KOfoegbu - I just heard back from our Synthetics Minion developers - they stated that at the moment the UI does not differentiate between CPM and regular Private Minion. As I assumed, this is since the CPM is still in Beta, this will definitely be updated & fixed before the CPM is GA’d however.


#11

Hi @RyanVeitch due to the synthetic minion going GA , I installed it in a lower environment , but when I tried to install it in pre production ( using 3 replicas ) , only one of the pods starts and the other two are in a crashloopbackoff. The third pod , though started does not get out of the checking minion health phase . The pods crashing show this error

synthetic-minion Dec 12, 2018, 10:16:13 PM 2018-12-13 03:16:13,400 - Service/dependency ‘DockerEngineHealthCheck’ is not healthy ‘Result{isHealthy=false, message=Docker ‘health check container’ exited with status ‘2’, timestamp=2018-12-13T03:16:12.811Z}’
synthetic-minion Dec 12, 2018, 10:16:12 PM 2018-12-13 03:16:12,811 - Docker ‘health check container’ exited with status ‘2’

That is the error it keeps throwing up . Can you provide any insight ?

Thanks.

Kenna


#12

Hey @KOfoegbu - I understand my colleague @Michel_L has opened up a support ticket for you. Please do let us know if you get a resolution here to share with the community.


#13

hi @RyanVeitch I was wondering , is there any restriction for deploying the minions on container optimized OS in GKE. I had initially deployed them on ubuntu node pools in GKE. However , when I install them on coos , it throws an error. Wonder if ubuntu is a pre req ?


#14

This is the error I am getting

" Invalid Docker configuration (API: ‘v1.35’ / Endpoint: ‘unix:///var/run/docker.sock’) and/or unable to contact the service. Is Docker running?"

However docker is running on the GKE node

“$ docker -v
Docker version 17.03.2-ce, build f5ec1e2”

We didn’t have this issue with the ubuntu nodes in GKE , so I was wondering if Container Optimized OS is the issue.

Thanks


#15

Hey @KOfoegbu - I’m not completely familiar with the containerised minions, however I don’t believe Ubuntu is a prerequisite. I’m going to follow up with our Synthetics team for confirmation on that.


#16

@KOfoegbu

I wanted to update the community here of progress on this question so far. The Container Optimized OS is based on Chromium OS which uses the Linux kernel upstream. Using an OS with a Linux kernel satisfies one of the options to meet the OS requirement. That being said, it appears the Container Optimized OS hardens /tmp to not be executable which is driving this error.

If you run sudo mount -o remount,exec /tmp on the CO-OS host to remount /tmp as executable, this will allow the minion to launch successfully.