NR- Synthetics - Private location setup - ssl issue

Hi Support,

I am getting the following SSLHandshake Error While trying to setup the private minion.

[root@ip-xx-xx-xx-xx centos]# docker run -e MINION_PRIVATE_LOCATION_KEY=NRSP-XXXXXXXXXX -v /tmp:/tmp/docker:rw -v /var/run/docker.sock:/var/run/docker.sock:rw quay.io/newrelic/synthetics-minion:latest
2020-07-01 03:20:31,448 - Minion won't be able to run: failed to fetch config for key 'NRSP-XXXXXXXXXX' from 'https://synthetics-horde.nr-data.net' ! javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
! at sun.security.ssl.Alerts.getSSLException(Alerts.java:198)
! at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1967)
! at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:331)
! at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:325)
! at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1688)
! at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:226)
! at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1082)
! at sun.security.ssl.Handshaker.process_record(Handshaker.java:1010)
! at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1079)
! at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1388)
! at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1416)
! at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1400)
! at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)
! at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
! at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
! at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:374)
! at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
! at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
! at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
! at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
! at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
! at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
! at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
! at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:221)
! at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165)
! at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:140)
! at com.newrelic.synthetics.minion.util.PrivateMinionUtils.getPrivateLocationMetadata(PrivateMinionUtils.java:184)
! at com.newrelic.synthetics.minion.cli.PrivateMinionLaunchCommandBase.augmentConfigurationViaHordeAPI(PrivateMinionLaunchCommandBase.java:179)
! at com.newrelic.synthetics.minion.cli.PrivateMinionLaunchCommandBase.run(PrivateMinionLaunchCommandBase.java:132)
! at com.newrelic.synthetics.minion.cli.PrivateMinionLaunchCommandBase.run(PrivateMinionLaunchCommandBase.java:40)
! at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:87)
! at io.dropwizard.cli.Cli.run(Cli.java:78)
! at io.dropwizard.Application.run(Application.java:93)
! at com.newrelic.synthetics.minion.MinionApplication.main(MinionApplication.java:98)
! Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
! at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:450)
! at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:317)
! at sun.security.validator.Validator.validate(Validator.java:262)
! at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330)
! at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237)
! at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132)
! at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1670)
! ... 29 common frames omitted
! Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
! at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
! at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
! at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
! at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:445)
! ... 35 common frames omitted

When I Curl the New Relic End point, getting the following response
[root@ip-xx-xx-xx-xx centos]# curl -X GET https://synthetics-horde.nr-data.net curl: (60) Peer’s Certificate issuer is not recognized.
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a “bundle”
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn’t adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you’d like to turn off curl’s verification of the certificate, use
the -k (or --insecure) option.

Could please look into this as it Could be either certificate issue from the server side or CACert update issuse.( SSL certificate of NR Endpoint - ‘https://synthetics-horde.nr-data.net’ seems to have broken or expired)

Also, it would be great if you can provide or point out a work around to access it in --insecure mode from docker run command

Thanks in advance

Hi @santhoshkumar1 - I have confirmed with a member of our Synthetics engineering team that there is no issue with the Horde certificate.

I’m going to get a ticket created for you so my colleagues can help look into this with you.

Hi @RyanVeitch,

Thanks for confirming that.

Could you please guide me if I have to modify the agent configuration or install cacerts?

Is it possible to by pass this SSL Verification for troubleshooting the issue and for checking the feasibility of this set up in a Private Network. If yes, Please do let me know

The CPM should just work when you download and run it. I’m not sure of any other requirement of installing CACerts…

As for bypassing SSL verification, I doubt that is possible, but my colleagues in that ticket can help you out with that question.

We were able to resolve this using the Proxy Environment variables:

  • MINION_API_PROXY
  • MINION_API_PROXY_AUTH
2 Likes