为Jenkins启动kubernetes吊舱颁发证书（托管在kubernetes集群之外）

Question

我们正在尝试使用Jenkins Kubernetes插件添加云代理。到kubernetes的连接可以工作（我已经通过测试连接验证了这一点。此外，当我的作业尝试启动时，pods容器会添加到集群中）。我正在我的pod模板中添加以下配置--pod容器在我的kubernetes引擎中启动。

问题-作业不运行，并不断创建新的豆荚和删除旧的豆荚。在正确的方向上需要一些帮助。我在网上搜索了一下，寻找是否有人有类似的问题或设置。似乎每个人都在k8s和云代理中托管詹金斯。

我认为问题是因为我们的詹金斯在我们的库伯内特斯星系团之外。

null

谷歌stackdriver日志错误-

SEVERE: Failed to connect to https://bflow.br.iq/tcpSlaveAgentListener/: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target`

`java.io.IOException: Failed to connect to https://bflow.br.iq/tcpSlaveAgentListener/: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:214)
    at hudson.remoting.Engine.innerRun(Engine.java:689)
    at hudson.remoting.Engine.run(Engine.java:514)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.ssl.Alert.createSSLException(Alert.java:131)
    at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
    at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
    at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
    at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
    at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
    at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:422)
    at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182)
    at sun.security.ssl.SSLTransport.decode(SSLTransport.java:149)
    at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1143)
    at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1054)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:394)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:167)
    at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:211)
    ... 2 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:456)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:323)
    at sun.security.validator.Validator.validate(Validator.java:271)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:315)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:223)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638)
    ... 16 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:451)
    ... 22 more

Answer 1

安装详细信息：

• Jenkins master作为docker容器在VM上运行。
• Jenkins在专用网络上运行。
• Kubernetes插件已安装。
• Kubernetes集群中创建的Jenkins命名空间、serviceAccount、Role和RoleBinding。
• 在托管节点和云>配置云>Kubernetes下，配置所需的详细信息。
  • Kubernetes URL
  • Kubernetes服务器证书密钥
  • Kubernetes命名空间
  • 凭据
  • WebSocket

  出现这个问题是因为最终用户必须使用带有HTTPS的Jenkins作为自签名证书。因此，当Kubernetes插件试图启动基本jenkins-inbound-agent容器时，它不会标识主Jenkins证书。因此，无法找到请求目标的有效证书路径错误。

  null

      $ openssl s_client -connect jenkins.my.domain.net:443 -showcerts > jenkins.crt
      depth=0 C = IN, ST = , L = Delhi, O = domain, OU = IT Operations, CN = *.jenkins.my.domain.net
      verify error:num=20:unable to get local issuer certificate
      verify return:1
      depth=0 C = IN, ST = , L = Delhi, O = domain, OU = IT Operations, CN = *.jenkins.my.domain.net
      verify error:num=21:unable to verify the first certificate
      verify return:1
      $ ls -lrth jenkins.crt
      -rw-rw-r--. 1 jenkins jenkins 3.3K Oct 20 09:52 jenkins.crt
      $
  

  COPY jenkins.crt /tmp/jenkins.crt
  RUN keytool -import -trustcacerts -keystore /opt/java/openjdk/jre/lib/security/cacerts -storepass ******* -noprompt -alias jenkins-master -file /tmp/jenkins.crt \
       && rm -Rf /tmp/jenkins.crt
  

  $ docker build . -t myregistery.company.net:5000/company/jenkins-agent:latest
  $ docker push myregistery.company.net:5000/company/jenkins-agent:latest
  

  • 在管理詹金斯>管理节点和云>配置云Kubernetes云详细信息
  中
  • 高级..>默认提供程序模板名称>设置值default-java
  • 展开pod模板
    • 将name设置为default-java
    • 将名称设置为JNLP
    • 将Docker图像设置为myregistery.company.net:5000/company/jenkins-agent:lates
    • 设置总是拉图像为真
    • 设置分配pseudo-tty为true

    在此之后，创建一个示例管道项目，并使用下面的代码在Kubernetes集群上动态测试运行jenkins-inbound-agent。

    pipeline {
      agent {
        kubernetes {
          yaml '''
            apiVersion: v1
            kind: Pod
            '''
        }
      }
      stages {
        stage('Run') {
          steps {
                sh 'date'
                sh 'ls -lrth'
            
          }
        }
      }
    }