Kubernetes安装系列之Node-Kubelet安装

这篇文章整理以下Node节点的kubelet的安装与设定方法，本文以脚本的方式进行固化，内容仍然放在github的easypack上。

整体操作

• https://blog.csdn.net/liumiaocn/article/details/88413428

kubelet的设定文件

[root@host131 shell]# cat /etc/k8s/kubelet.conf 
KUBELET_OPTS="--logtostderr=true \
--v=4 \
--log-dir=/var/log/kubernetes \
--root-dir=/var/lib/kubelet \
--cert-dir=/etc/ssl/k8s \
--fail-swap-on=false \
--hostname-override=192.168.163.131 \
--bootstrap-kubeconfig=/etc/ssl/k8s/bootstrap.kubeconfig \
--kubeconfig=/etc/k8s/kubelet.kubeconfig \
--config=/etc/k8s/kubelet-config.yaml \
--pod-infra-container-image=gcr.io/google_containers/pause-amd64:3.1 \
--allow-privileged=true \
--event-qps=0 \
--kube-api-qps=1000 \
--kube-api-burst=2000 \
--registry-qps=0 \
--image-pull-progress-deadline=30m"
[root@host131 shell]#

config设定文件

从1.10开始，很多参数都需要在config指定的文件中进行设定，设定示例如下

[root@host131 shell]# cat /etc/k8s/kubelet-config.yaml
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: true
  x509:
    clientCAFile: "/etc/ssl/ca/ca.pem"
authorization:
  mode: Webhook
clusterDomain: "cluster.local"
clusterDNS:
  - "10.0.0.2"
podCIDR: "172.200.0.0/16"
maxPods: 2000
serializeImagePulls: false
hairpinMode: promiscuous-bridge
cgroupDriver: cgroupfs
runtimeRequestTimeout: "15m"
rotateCertificates: true
serverTLSBootstrap: true
readOnlyPort: 0
port: 10250
address: "192.168.163.131"
[root@host131 shell]# 

Systemd服务配置文件

[root@host131 shell]# cat /usr/lib/systemd/system/kubelet.service 
[Unit]
Description=Kubernetes Kubelet Service
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=-/etc/k8s/kubelet.conf
ExecStart=/usr/local/bin/kubelet $KUBELET_OPTS
Restart=always
RestartSec=5
StartLimitInterval=0

[Install]
WantedBy=multi-user.target
[root@host131 shell]#

脚本示例

[root@host131 shell]# cat step8-2-install-kubelet.sh 
#!/bin/sh

. ./install.cfg

echo -e "\n##  kubelet service"
systemctl stop kubelet 2>/dev/null

mkdir -p ${ENV_KUBE_DIR_BIN} ${ENV_KUBE_DIR_ETC} ${ENV_KUBE_OPT_LOG_DIR} ${ENV_KUBELET_DIR_WORKING}
chmod 755 ${ENV_HOME_K8S}/*
cp -p ${ENV_HOME_K8S}/kubelet ${ENV_KUBE_DIR_BIN}
if [ $? -ne 0 ]; then
  echo "please check kubelet binary files existed in ${ENV_HOME_K8S}/ or not"
  exit 
fi

# create kubelet configuration file
cat >${ENV_KUBE_DIR_ETC}/${ENV_KUBE_KUBELET_ETC} <<EOF
KUBELET_OPTS="--logtostderr=${ENV_KUBE_OPT_LOGTOSTDERR} \\
--v=${ENV_KUBE_OPT_LOG_LEVEL} \\
--log-dir=${ENV_KUBE_OPT_LOG_DIR} \\
--root-dir=${ENV_KUBELET_DIR_WORKING} \\
--cert-dir=${ENV_SSL_K8S_DIR} \\
--fail-swap-on=${ENV_KUBELET_OPT_FAIL_SWAP_ON} \\
--hostname-override=${ENV_KUBE_NODE_HOSTNAME} \\
--bootstrap-kubeconfig=${ENV_SSL_K8S_DIR}/${ENV_KUBECONFIG_BOOTSTRAP} \\
--kubeconfig=${ENV_KUBE_DIR_ETC}/${ENV_KUBELET_KUBECONFIG} \\
--config=${ENV_KUBE_DIR_ETC}/${ENV_KUBELET_OPT_CONFIG} \\
--pod-infra-container-image=${ENV_KUBE_OPT_PAUSE} \\
--allow-privileged=${ENV_KUBE_OPT_ALLOW_PRIVILEGE} \\
--event-qps=${ENV_KUBELET_OPT_EVENT_QPS} \\
--kube-api-qps=${ENV_KUBELET_OPT_KPI_QPS} \\
--kube-api-burst=${ENV_KUBELET_OPT_API_BRUST} \\
--registry-qps=${ENV_KUBELET_OPT_REG_QPS} \\
--image-pull-progress-deadline=${ENV_KUBELET_OPT_IMG_PULL_DEADLINE}"
EOF

# create kubelet config yaml file for config option
cat >${ENV_KUBE_DIR_ETC}/${ENV_KUBELET_OPT_CONFIG} <<EOF 
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: ${ENV_KUBELET_CONFIG_OPT_ANONYMOUS}
  webhook:
    enabled: ${ENV_KUBELET_CONFIG_OPT_WEBHOOK}
  x509:
    clientCAFile: "${ENV_SSL_CA_DIR}/${ENV_SSL_FILE_CA_PEM}"
authorization:
  mode: ${ENV_KUBELET_CONFIG_OPT_MODE}
clusterDomain: "${ENV_KUBELET_CONFIG_OPT_CLUSTER_DOMAIN}"
clusterDNS:
  - "${ENV_KUBELET_CONFIG_OPT_CLUSTER_DNS}"
podCIDR: "${ENV_KUBE_OPT_CLUSTER_IP_RANGE}"
maxPods: ${ENV_KUBELET_CONFIG_OPT_MAXPODS}
serializeImagePulls: ${ENV_KUBELET_CONFIG_OPT_SERIALIZE_IMG_PULL}
hairpinMode: ${ENV_KUBELET_CONFIG_OPT_HAIRPIN}
cgroupDriver: ${ENV_KUBELET_CONFIG_OPT_CGROUP_DRIVER}
runtimeRequestTimeout: "${ENV_KUBELET_CONFIG_OPT_REQUEST_TMO}"
rotateCertificates: ${ENV_KUBELET_CONFIG_OPT_ROTATE_CERT}
serverTLSBootstrap: ${ENV_KUBELET_CONFIG_OPT_TLS_BOOTSTRAP}
readOnlyPort: ${ENV_KUBELET_CONFIG_OPT_READONLY_PORT}
port: ${ENV_KUBELET_CONFIG_OPT_PORT}
address: "${ENV_KUBE_NODE_HOSTNAME}"
EOF

# Create the kubelet service.
cat >${ENV_KUBE_KUBELET_SERVICE} <<EOF
[Unit]
Description=Kubernetes Kubelet Service
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=${ENV_KUBELET_DIR_WORKING}
EnvironmentFile=-${ENV_KUBE_DIR_ETC}/${ENV_KUBE_KUBELET_ETC}
ExecStart=${ENV_KUBE_DIR_BIN}/kubelet \$KUBELET_OPTS
Restart=always
RestartSec=5
StartLimitInterval=0

[Install]
WantedBy=multi-user.target
EOF

echo -e "\n##  daemon reload service "
systemctl daemon-reload
echo -e "\n##  start kubelet service "
systemctl start kubelet
echo -e "\n##  enable kubelet service " 
systemctl enable kubelet
echo -e "\n##  check  kubelet status"
systemctl status kubelet


echo
echo -e "\n##  get csr information"
kubectl get csr

echo -e "##  kubectl get nodes "
kubectl get nodes -o wide
[root@host131 shell]# 

执行示例

为了执行方便，在这些脚本外边在包一层，统一使用如下脚本进行管理

[root@host131 shell]# sh all-k8s-mgnt.sh install kubelet
## ACTION: install  Service: kubelet begins ...
2019/03/24 20:06:26 [INFO] generate received request
2019/03/24 20:06:26 [INFO] received CSR
2019/03/24 20:06:26 [INFO] generating key: rsa-2048
2019/03/24 20:06:26 [INFO] encoded CSR
2019/03/24 20:06:26 [INFO] signed certificate with serial number 100213249864002235085413152226418981333611978799
2019/03/24 20:06:26 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
/etc/ssl/k8s/cert-kubeproxy-key.pem  /etc/ssl/k8s/cert-kubeproxy.pem
Cluster "kubernetes" set.
User "kubelet-bootstrap" set.
Context "default" created.
Switched to context "default".
Cluster "kubernetes" set.
User "kube-proxy" set.
Context "default" created.
Switched to context "default".
clusterrolebinding.rbac.authorization.k8s.io/kubelet-bootstrap created

##  kubelet service

##  daemon reload service 

##  start kubelet service 

##  enable kubelet service 

##  check  kubelet status
● kubelet.service - Kubernetes Kubelet Service
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-03-24 20:06:29 CST; 388ms ago
     Docs: https://github.com/GoogleCloudPlatform/kubernetes
 Main PID: 1134 (kubelet)
   CGroup: /system.slice/kubelet.service
           ├─1134 /usr/local/bin/kubelet --logtostderr=true --v=4 --log-dir=/var/log/kubernetes --root-dir=/var/lib/kubelet --cert-dir=/etc/ssl/k8s -...
           └─1160 systemd-run --description=Kubernetes systemd probe --scope true

Mar 24 20:06:29 host131 kubelet[1134]: I0324 20:06:29.272697    1134 flags.go:33] FLAG: --file-check-frequency="20s"
Mar 24 20:06:29 host131 kubelet[1134]: I0324 20:06:29.272704    1134 flags.go:33] FLAG: --global-housekeeping-interval="1m0s"
Mar 24 20:06:29 host131 kubelet[1134]: I0324 20:06:29.272712    1134 flags.go:33] FLAG: --hairpin-mode="promiscuous-bridge"
Mar 24 20:06:29 host131 kubelet[1134]: I0324 20:06:29.272719    1134 flags.go:33] FLAG: --healthz-bind-address="127.0.0.1"
Mar 24 20:06:29 host131 kubelet[1134]: I0324 20:06:29.272726    1134 flags.go:33] FLAG: --healthz-port="10248"
Mar 24 20:06:29 host131 kubelet[1134]: I0324 20:06:29.272733    1134 flags.go:33] FLAG: --help="false"
Mar 24 20:06:29 host131 kubelet[1134]: I0324 20:06:29.272739    1134 flags.go:33] FLAG: --host-ipc-sources="[*]"
Mar 24 20:06:29 host131 kubelet[1134]: I0324 20:06:29.272755    1134 flags.go:33] FLAG: --host-network-sources="[*]"
Mar 24 20:06:29 host131 kubelet[1134]: I0324 20:06:29.272762    1134 flags.go:33] FLAG: --host-pid-sources="[*]"
Mar 24 20:06:29 host131 kubelet[1134]: I0324 20:06:29.272774    1134 flags.go:33] FLAG: --hostname-override="192.168.163.131"


##  get csr information
No resources found.
##  kubectl get nodes 
No resources found.
## ACTION: install  Service: kubelet ends  ...

[root@host131 shell]#

设定之后可以进行bootstrap的机制会自动发出csr请求，而通过kubectl certificate approve则可手动发行证书。

[root@host131 shell]# kubectl certificate approve node-csr-ySkXjxhHO0w8zy39-YXzSSVxDtwnYJUCuFxhseDPoLk
certificatesigningrequest.certificates.k8s.io/node-csr-ySkXjxhHO0w8zy39-YXzSSVxDtwnYJUCuFxhseDPoLk approved
[root@host131 shell]# 
[root@host131 shell]# kubectl get csr
NAME                                                   AGE   REQUESTOR                     CONDITION
node-csr-ySkXjxhHO0w8zy39-YXzSSVxDtwnYJUCuFxhseDPoLk   40s   kubelet-bootstrap             Approved,Issued
[root@host131 shell]# 

再次确认get nodes，则可以看到此节点已被master所识别出来。

[root@host131 shell]# kubectl get nodes
NAME              STATUS   ROLES    AGE   VERSION
192.168.163.131   Ready    <none>   15s   v1.13.4
[root@host131 shell]# kubectl get nodes -o wide
NAME              STATUS   ROLES    AGE   VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION          CONTAINER-RUNTIME
192.168.163.131   Ready    <none>   19s   v1.13.4   192.168.163.131   <none>        CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://17.3.2
[root@host131 shell]#