Kubernetes CoreDNS吊舱无休止地重新启动

Question

null

null

Failed to list *v1.Namespace: Get https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: connect: no route to host

default via 172.16.0.1 dev eth0 proto static metric 100 
10.1.0.0/24 dev eth1 proto kernel scope link src 10.1.0.202 metric 101 
10.244.0.0/24 via 10.244.0.0 dev flannel.1 onlink 
10.244.1.0/24 dev docker0 proto kernel scope link src 10.244.1.1 
10.244.1.0/24 dev cni0 proto kernel scope link src 10.244.1.1 
10.244.2.0/24 via 10.244.2.0 dev flannel.1 onlink 
172.16.0.0/16 dev eth0 proto kernel scope link src 172.16.0.202 metric 100

null

>

将Kubernetes降级为1.11.3-0。

使用kubeadm init-apiserver-advise-address=172.16.0.201-pod-network-cidr=10.244.0.0/16重新初始化了Kubernetes，因为服务器有另一个外部IP，无法通过其他主机访问，而Kubernetes倾向于选择该IP作为API服务器IP。--pod-network-cidr由Flannel授权。

初始化后没有连接节点的结果iptables-l输出

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
KUBE-EXTERNAL-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes externally-visible service portals */
KUBE-FIREWALL  all  --  anywhere             anywhere            

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
KUBE-FORWARD  all  --  anywhere             anywhere             /* kubernetes forwarding rules */
DOCKER-USER  all  --  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
KUBE-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes service portals */
KUBE-FIREWALL  all  --  anywhere             anywhere            

Chain DOCKER-USER (1 references)
target     prot opt source               destination         
RETURN     all  --  anywhere             anywhere            

Chain KUBE-EXTERNAL-SERVICES (1 references)
target     prot opt source               destination         

Chain KUBE-FIREWALL (2 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere             /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000

Chain KUBE-FORWARD (1 references)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere             /* kubernetes forwarding rules */ mark match 0x4000/0x4000

Chain KUBE-SERVICES (1 references)
target     prot opt source               destination         
REJECT     udp  --  anywhere             10.96.0.10           /* kube-system/kube-dns:dns has no endpoints */ udp dpt:domain reject-with icmp-port-unreachable
REJECT     tcp  --  anywhere             10.96.0.10           /* kube-system/kube-dns:dns-tcp has no endpoints */ tcp dpt:domain reject-with icmp-port-unreachable

看起来API服务器已按其应有的方式部署

$ kubectl get svc kubernetes -o=yaml
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: 2018-10-25T06:58:46Z
  labels:
    component: apiserver
    provider: kubernetes
  name: kubernetes
  namespace: default
  resourceVersion: "6"
  selfLink: /api/v1/namespaces/default/services/kubernetes
  uid: 6b3e4099-d823-11e8-8264-a6f3f1f622f3
spec:
  clusterIP: 10.96.0.1
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 6443
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

然后我将法兰绒网络pod应用于

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Failed to list *v1.Endpoints: Get https://10.96.0.1:443/api/v1/endpoints?limit=500\u0026resourceVersion=0: dial tcp 10.96.0.1:443: connect: no route to host

Answer 1

这基本上是说coredns pod不能与Kube-Apiserver对话。kube-apiserver通过以下环境变量在pod中公开:kubernetes_service_host=10.96.0.1和kubernetes_service_port_https=443

我相信您发布的路由是主机上的路由，因为这是您在pod容器中运行ip routes时得到的结果：

root@xxxx-xxxxxxxxxx-xxxxx:/# ip route
default via 169.254.1.1 dev eth0
169.254.1.1 dev eth0  scope link
root@xxxx-xxxxxxxxxx-xxxxx:/#

在任何情况下，您都不会看到10.96.0.1，因为它是使用iptables在集群中公开的。那地址是什么？缺省命名空间中的服务称为Kubernetes。该服务的clusterIP是10.96.0.1，它正在侦听端口443，它还映射到targetport6443，而这正是您的kube-apiserver正在运行的地方。

因为您可以部署pods等，所以看起来kube-apiserver没有关闭，这不是您的问题。所以很可能您缺少了该服务（或者某个iptable规则不允许您连接到它）。你可以在这里看到，例如：

$ kubectl get svc kubernetes
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   92d

完整输出如下所示：

$ kubectl get svc kubernetes -o=yaml
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: 2018-07-23T21:10:22Z
  labels:
    component: apiserver
    provider: kubernetes
  name: kubernetes
  namespace: default
  resourceVersion: "24"
  selfLink: /api/v1/namespaces/default/services/kubernetes
  uid: xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx
spec:
  clusterIP: 10.96.0.1
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 6443
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {} 

因此，如果你错过了它，你可以像这样创建它：

cat <<EOF
apiVersion: v1
kind: Service
metadata:
  labels:
    component: apiserver
    provider: kubernetes
  name: kubernetes
  namespace: default
spec:
  clusterIP: 10.96.0.1
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 6443
  sessionAffinity: None
  type: ClusterIP
EOF | kubectl apply -f -