K3S Kubernetes集群中Traefik入口控制器的AWS证书解析器，带有现有的AWS HTTPS负载平衡器

Question

1. 我有AWS K3S库伯内特斯集群
2. 我有AWS负载均衡器
3. 我已注册域名
4. 我已经注册了AWS证书
5. 我为我的域和AWS负载均衡器DNS名称创建了CNAME记录
6. 我在AWS K3S库伯内特斯集群上安装了Traefik入口控制器
7. 我将用户管理和whoami服务部署到AWS K3S库伯内特斯集群
8. 我创建了带有用户管理和whoami路径的Traefik入口

问题是：

如何使用Ingress Traefik Controller将托管在我的域上的AWS负载平衡器连接到K3s上的服务？

或者换句话说：

如何适应下面描述的“traefik服务”或“traefik部署”，以便在我注册的域中使用AWS证书解析器？

或任何关于如何使用

• AWS负载html" target="_blank">均衡器、AWS目标组、AWS安全组，使用Terraform文件创建
• 与部署到K3S库伯内特斯集群的Traefik入口控制器和Traefik入口路由相结合，使用AWS证书解析。

我目前无法通过AWS负载平衡器连接到我的服务。返回以下错误：

404 Page Not Found

502 Bad Gateway

以下是URL的示例，我尝试了：

https://keycloak.skycomposer.net/usermgmt
https://keycloak.skycomposer.net/whoami

我为“usermgmt”和“whoami”kubernetes服务设置了相应的入口路径。

以下是更多信息：

• 我使用负载平衡器在AWS中创建了K3S Kubernetes集群

这些是我的地形文件：https://github.com/skyglass/user-management/tree/master/terraform

K3S集群部署到EC2实例（请参阅“userdata.tpl”脚本）

我禁用了Traefik入口控制器部署，以便稍后部署它。

• 我在这里找到了如何将“Traefik”安装到K3S Kubernetes集群的示例：https://github.com/sleighzy/k3s-traefik-v2-kubernetes-crd

不幸的是，这个示例使用“godaddy”证书解析器，但我的域是用AWS路由53注册的，我使用的是AWS证书管理器。

以下是我尝试调整的“traefik服务”和“traefik部署”文件：

traefik服务：

---
apiVersion: v1
kind: Service
metadata:
  name: traefik
  namespace: kube-system

spec:
  # The targetPort entries are required as the Traefik container is listening on ports > 1024
  # so that the container can be run as a non-root user and they can bind to these ports.
  # Traefik is still accessed over 80 and 443 on the host, but the service routes the traffic
  # to ports 8080 and 8443 on the container.
  ports:
    - protocol: TCP
      name: web
      port: 80
      targetPort: 8080
    - protocol: TCP
      name: websecure
      port: 443
      targetPort: 8443
    - protocol: TCP
      name: admin
      port: 8080
      targetPort: 9080
  selector:
    app: traefik
  # Set externalTrafficPolicy to Local so that all external traffic intended for
  # the Traefik pod goes directly to that local node. If the default of Cluster is
  # used instead then the client source IP address is lost, and may hop between nodes.
  externalTrafficPolicy: Local
  type: LoadBalancer

traefik部署：

---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: kube-system
  name: traefik-ingress-controller

---
kind: Deployment
apiVersion: apps/v1
metadata:
  namespace: kube-system
  name: traefik
  labels:
    app: traefik

spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      containers:
        - name: traefik
          image: traefik:v2.4
          args:
            - --api.dashboard=true
            - --ping=true
            - --accesslog
            - --entrypoints.traefik.address=:9080
            - --entrypoints.web.address=:8080
            - --entrypoints.websecure.address=:8443
            # Uncomment the below lines to redirect http requests to https.
            # This specifies the port :443 and not the https entrypoint name for the
            # redirect as the service is listening on port 443 and directing traffic
            # to the 8443 target port. If the entrypoint name "websecure" was used,
            # instead of "to=:443", then the browser would be redirected to port 8443.
            - --entrypoints.web.http.redirections.entrypoint.to=:443
            - --entrypoints.web.http.redirections.entrypoint.scheme=https
            - --providers.kubernetescrd
            - --providers.kubernetesingress
            - --certificatesresolvers.myresolver.acme.tlschallenge=true
            - --certificatesresolvers.myresolver.acme.email=[e-mail]
            - --certificatesresolvers.myresolver.acme.storage=/etc/traefik/certs/acme.json          
            # Please note that this is the staging Let's Encrypt server.
            # Once you get things working, you should remove that whole line altogether.
            # - --certificatesresolvers.godaddy.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory
            - --log
            - --log.level=INFO
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /ping
              port: 9080
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 3
          resources:
            limits:
              memory: '100Mi'
              cpu: '1000m'
          ports:
            # The Traefik container is listening on ports > 1024 so the container
            # can be run as a non-root user and they can bind to these ports.
            - name: web
              containerPort: 8080
            - name: websecure
              containerPort: 8443
            - name: admin
              containerPort: 9080
          volumeMounts:
            - name: certificates
              mountPath: /etc/traefik/certs
      # volumes:
      #   - name: certificates
      #     persistentVolumeClaim:
      #       claimName: traefik-certs-pvc              
      volumes:
        - name: certificates
          hostPath:
            path: "/Users/dddd/git/aws/letsencrypt:/etc/traefik/certs"

在此处查看其他文件：https://github.com/sleighzy/k3s-traefik-v2-kubernetes-crd

理想情况下，应该有这样的解决方案：

apiVersion: v1
kind: Service
metadata:
  name: traefik-proxy
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:REGION:ACCOUNTID:certificate/CERT-ID"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
spec:
  type: LoadBalancer
  selector:
    app: traefik-proxy
    tier: proxy
  ports:
  - port: 443
    targetPort: 80

在这个解决方案中，我只需提供我的AWS证书ARN，traefik ingress controller将完成其他所有工作。

本文介绍了类似的解决方案：

https://www.ronaldjamesgroup.com/blog/getting-started-with-traefik

但是，不幸的是，这个解决方案对我也不起作用，我尝试了它，但没有成功。

返回以下错误：

404 Page Not Found

502 Bad Gateway

当我尝试为我的域输入路由路径时：

https://keycloak.skycomposer.net/usermgmt
https://keycloak.skycomposer.net/whoami

Answer 1

在尝试了几个选项后，我终于找到了解决方案：https://github.com/skyglass-examples/aws-k3s-traefik

1. 我用Terraform创建了AWS负载平衡器和K3S集群

这是我的Traefik入口控制器清单文件：

traefik部署。yaml：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system


---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: traefik-proxy
  namespace: kube-system     
  labels:
    app: traefik-proxy
    tier: proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik-proxy
      tier: proxy
  template:
    metadata:
      labels:
        app: traefik-proxy
        tier: proxy
    spec:
      serviceAccountName: traefik-ingress-controller   
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik:v1.2.0-rc1-alpine
        name: traefik-proxy
        ports:
        - containerPort: 80
          hostPort: 80
          name: traefik-proxy
        - containerPort: 8080
          name: traefik-ui
        args:
          - --web
          - --kubernetes     

traefik服务。yaml：

apiVersion: v1
kind: Service
metadata:
  name: traefik-proxy
  namespace: kube-system   
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-west-1:dddddddddd"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-internal: "0.0.0.0/0"
    service.beta.kubernetes.io/aws-load-balancer-type: "alb"
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local  
  selector:
    app: traefik-proxy
    tier: proxy
  ports:
  - port: 443
    targetPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system     
spec:
  selector:
    app: traefik-proxy
    tier: proxy
  ports:
  - port: 80
    targetPort: 8080

traefik入口。yaml：

apiVersion: networking.k8s.io/v1beta1
kind: IngressClass
metadata:
  name: traefik-lb
spec:
  controller: traefik.io/ingress-controller

---
apiVersion: "networking.k8s.io/v1beta1"
kind: "Ingress"
metadata:
  name: "traefik-usermgmt-ingress"
spec:
  ingressClassName: "traefik-lb"
  rules:
  - host: "keycloak.skycomposer.net"
    http:
      paths:
      - path: "/usermgmt"
        backend:
          serviceName: "usermgmt"
          servicePort: 80


---
apiVersion: "networking.k8s.io/v1beta1"
kind: "Ingress"
metadata:
  name: "traefik-whoami-ingress"
spec:
  ingressClassName: "traefik-lb"
  rules:
  - host: "keycloak.skycomposer.net"
    http:
      paths:
      - path: "/whoami"
        backend:
          serviceName: "whoami"
          servicePort: 80

请参见此处的完整代码：https://github.com/skyglass-examples/aws-k3s-traefik

代码包括：

1. AWS负载平衡器和K3S Kubernetes群集的terraform文件
2. 我部署到K3S的docker容器之一的源代码
3. Traefik Ingress Controller、2个kubernetes服务和Traefik Ingress的kubernetes清单文件，在注册域上通过安全HTTPS连接公开这些服务
4. 将AWS证书ARN替换为您证书的相应ARN
5. 将“skycomposer.net”替换为您的域名（请参阅自述文件中的更多详细信息：https://github.com/skyglass-examples/aws-k3s-traefik)