ubuntu20.04下使用juju+maas环境部署k8s-3-处理Missing flannel resource
万俟超
参考文档：
Kubernetes documentation
使用 Graylog 和 Prometheus 监视 Kubernetes 集群
接上节：
juju status --relations
Model Controller Cloud/Region Version SLA Timestamp
k8s maas-controller mymaas/default 2.8.10 unsupported 17:19:05+08:00
App Version Status Scale Charm Store Channel Rev OS Message
containerd go1.13.8 active 5 containerd local 0 ubuntu Container runtime available
easyrsa 3.0.1 active 1 easyrsa local 0 ubuntu Certificate Authority connected.
etcd 3.4.5 active 3 etcd local 0 ubuntu Healthy with 3 known peers
flannel blocked 5 flannel local 0 ubuntu Missing flannel resource.
会发现flannel状态不对，为Missing flannel resource.，且没返回版本号。
开始以为是flannel的源在gw外，所以开始研读相关文档。
Flannel #558
描述
它是一个通用的覆盖网络，可用作现有软件定义网络解决方案的简单替代方案。
Flannel charm
Flannel 是一个虚拟网络，它为每个主机提供一个子网，用于容器运行时。
此 Charm 会将 flannel 部署为后台服务，并在实现 kubernetes-cni 接口的任何主体 Charm 上配置 CNI 以与 flannel 一起使用。
这种魅力与 Charmed Kubernetes 的组件一起维护。有关完整信息，请访问 Charmed Kubernetes 官方文档。
在其中的flannel.py
@when_not('flannel.binaries.installed')
def install_flannel_binaries():
''' Unpack the Flannel binaries. '''
try:
resource_name = 'flannel-{}'.format(arch())
archive = resource_get(resource_name)
except Exception:
message = 'Error fetching the flannel resource.'
log(message)
status.blocked(message)
return
if not archive:
message = 'Missing flannel resource.'
log(message)
status.blocked(message)
return
filesize = os.stat(archive).st_size
if filesize < 1000000:
message = 'Incomplete flannel resource'
log(message)
status.blocked(message)
return
看来没找到相关下载的资源造成的。
接着看文档：
在其中的的build-flannel-resources.sh中，发现如下段落：
FLANNEL_VERSION=${FLANNEL_VERSION:-"v0.11.0"}
ETCD_VERSION=${ETCD_VERSION:-"v2.3.7"}
ARCH=${ARCH:-"amd64 arm64 s390x"}
build_script_commit="$(git show --oneline -q)"
temp_dir="$(readlink -f build-flannel-resources.tmp)"
rm -rf "$temp_dir"
mkdir "$temp_dir"
(cd "$temp_dir"
git clone https://github.com/coreos/flannel.git flannel \
--branch "$FLANNEL_VERSION" \
--depth 1
git clone https://github.com/coreos/etcd.git etcd \
--branch "$ETCD_VERSION" \
--depth 1
这段是git clone 了flannel和etcd资源
git clone https://github.com/coreos/flannel.git，版本0.11.0
git clone https://github.com/coreos/etcd.git，版本2.3.7
开始想法是是直接下载，根据前文多节点OpenStack Charms 部署指南0.0.1.dev299–16–OpenStack基础架构高可用The easyrsa resource is missing. .中的办法：
wget https://github.com/flannel-io/flannel/releases/download/v0.11.0/flannel-v0.11.0-linux-amd64.tar.gz
juju attach flannel flannel=./flannel-v0.11.0-linux-amd64.tar.gz
output is:
ERROR failed to upload resource “flannel”: Put “https://10.0.0.155:17070/model/9fc2675f-3030-4102-83aa-6c76fd182926/applications/flannel/resources/flannel”: write tcp 10.0.0.3:56746->10.0.0.155:17070: write: connection reset by peer
看来这个办法有问题。
判断了下，既然是能下载下来，说明不是源的问题，估计是本地部署的时候，有了没有执行这个sh。
为了判断，干脆重新charm store部署了下。（其中kubernetes-master和easyrsa是 local部署。）
vim bundle.yaml
description: A highly-available, production-grade Kubernetes cluster.
series: focal
services:
containerd:
annotations:
gui-x: '475'
gui-y: '800'
charm: cs:~containers/containerd-130
resources: {}
easyrsa:
annotations:
gui-x: '90'
gui-y: '420'
charm: /root/charmed-kubernetes-679/easyrsa
constraints: root-disk=8G
num_units: 1
resources:
easyrsa: 5
etcd:
annotations:
gui-x: '800'
gui-y: '420'
charm: cs:~containers/etcd-594
constraints: root-disk=8G
num_units: 3
options:
channel: 3.4/stable
resources:
core: 0
etcd: 3
snapshot: 0
flannel:
annotations:
gui-x: '475'
gui-y: '605'
charm: cs:~containers/flannel-558
resources:
flannel-amd64: 761
flannel-arm64: 758
flannel-s390x: 745
kubeapi-load-balancer:
annotations:
gui-x: '450'
gui-y: '250'
charm: cs:~containers/kubeapi-load-balancer-798
constraints: mem=4G root-disk=8G
expose: true
num_units: 1
resources: {}
kubernetes-master:
annotations:
gui-x: '800'
gui-y: '850'
charm: /root/charmed-kubernetes-679/kubernetes-master
constraints: cores=2 mem=4G root-disk=16G
num_units: 2
options:
channel: 1.21/stable
resources:
cdk-addons: 0
core: 0
kube-apiserver: 0
kube-controller-manager: 0
kube-proxy: 0
kube-scheduler: 0
kubectl: 0
kubernetes-worker:
annotations:
gui-x: '90'
gui-y: '850'
charm: cs:~containers/kubernetes-worker-768
constraints: cores=4 mem=4G root-disk=16G
expose: true
num_units: 3
options:
channel: 1.21/stable
resources:
cni-amd64: 797
cni-arm64: 788
cni-s390x: 800
core: 0
kube-proxy: 0
kubectl: 0
kubelet: 0
relations:
- - kubernetes-master:kube-api-endpoint
- kubeapi-load-balancer:apiserver
- - kubernetes-master:loadbalancer
- kubeapi-load-balancer:loadbalancer
- - kubernetes-master:kube-control
- kubernetes-worker:kube-control
- - kubernetes-master:certificates
- easyrsa:client
- - etcd:certificates
- easyrsa:client
- - kubernetes-master:etcd
- etcd:db
- - kubernetes-worker:certificates
- easyrsa:client
- - kubernetes-worker:kube-api-endpoint
- kubeapi-load-balancer:website
- - kubeapi-load-balancer:certificates
- easyrsa:client
- - flannel:etcd
- etcd:db
- - flannel:cni
- kubernetes-master:cni
- - flannel:cni
- kubernetes-worker:cni
- - containerd:containerd
- kubernetes-worker:container-runtime
- - containerd:containerd
- kubernetes-master:container-runtime
juju deploy ./bundle.yaml
juju status
Model Controller Cloud/Region Version SLA Timestamp
k8s maas-controller mymaas/default 2.8.10 unsupported 16:49:34+08:00
App Version Status Scale Charm Store Channel Rev OS Message
containerd go1.13.8 active 5 containerd charmstore 130 ubuntu Container runtime available
easyrsa 3.0.1 active 1 easyrsa local 0 ubuntu Certificate Authority connected.
etcd 3.4.5 active 3 etcd charmstore 594 ubuntu Healthy with 3 known peers
flannel 0.11.0 active 5 flannel charmstore 558 ubuntu Flannel subnet 10.1.47.1/24
kubeapi-load-balancer 1.18.0 active 1 kubeapi-load-balancer charmstore 798 ubuntu Loadbalancer ready.
kubernetes-master 1.21.1 waiting 2 kubernetes-master local 0 ubuntu Waiting for 3 kube-system pods to start
kubernetes-worker 1.21.1 active 3 kubernetes-worker charmstore 768 ubuntu Kubernetes worker running.
rsyslog-forwarder-ha unknown 0 rsyslog-forwarder-ha charmstore 20 ubuntu
Unit Workload Agent Machine Public address Ports Message
easyrsa/0* active idle 0 10.0.3.189 Certificate Authority connected.
etcd/0* active idle 1 10.0.3.200 2379/tcp Healthy with 3 known peers
etcd/1 active idle 2 10.0.3.201 2379/tcp Healthy with 3 known peers
etcd/2 active idle 3 10.0.3.204 2379/tcp Healthy with 3 known peers
kubeapi-load-balancer/0* active idle 4 10.0.3.208 443/tcp Loadbalancer ready.
kubernetes-master/0 waiting idle 5 10.0.3.202 6443/tcp Waiting for 3 kube-system pods to start
containerd/4 active idle 10.0.3.202 Container runtime available
flannel/4 active idle 10.0.3.202 Flannel subnet 10.1.22.1/24
kubernetes-master/1* waiting idle 6 10.0.3.207 6443/tcp Waiting for 3 kube-system pods to start
containerd/2 active idle 10.0.3.207 Container runtime available
flannel/2 active idle 10.0.3.207 Flannel subnet 10.1.18.1/24
kubernetes-worker/0* active idle 7 10.0.3.203 80/tcp,443/tcp Kubernetes worker running.
containerd/0* active idle 10.0.3.203 Container runtime available
flannel/0* active idle 10.0.3.203 Flannel subnet 10.1.47.1/24
kubernetes-worker/1 active idle 8 10.0.3.206 80/tcp,443/tcp Kubernetes worker running.
containerd/3 active idle 10.0.3.206 Container runtime available
flannel/3 active idle 10.0.3.206 Flannel subnet 10.1.4.1/24
kubernetes-worker/2 active idle 9 10.0.3.205 80/tcp,443/tcp Kubernetes worker running.
containerd/1 active idle 10.0.3.205 Container runtime available
flannel/1 active idle 10.0.3.205 Flannel subnet 10.1.69.1/24
Machine State DNS Inst id Series AZ Message
0 started 10.0.3.189 busy-raptor focal default Deployed
1 started 10.0.3.200 crisp-swift focal default Deployed
2 started 10.0.3.201 vital-tick focal default Deployed
3 started 10.0.3.204 stable-dory focal default Deployed
4 started 10.0.3.208 upward-ibex focal default Deployed
5 started 10.0.3.202 ideal-oyster focal default Deployed
6 started 10.0.3.207 safe-goat focal default Deployed
7 started 10.0.3.203 glad-hen focal default Deployed
8 started 10.0.3.206 cool-aphid focal default Deployed
9 started 10.0.3.205 epic-moose focal default Deployed
看来的确是本地部署产生的问题，下次如果有机会可以在本地部署后，执行下build-flannel-resources.sh试试。
类似资料：