kubernetes+virtual-kubelet对接openstack zun

商同

2023-12-01

1、环境介绍

在一个OpenStack Train集群上创建4个虚拟机，运行k8s集群
在另外一个OpenStac Train集群上运行zun

# kubectl get nodes -o wide
NAME               STATUS   ROLES    AGE    VERSION                                INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION           CONTAINER-RUNTIME
k8s-master1        Ready    master   3d8h   v1.17.0                                192.168.1.102   <none>        CentOS Linux 7 (Core)   3.10.0-1062.el7.x86_64   docker://19.3.0
k8s-master2        Ready    master   3d8h   v1.17.0                                192.168.1.169   <none>        CentOS Linux 7 (Core)   3.10.0-1062.el7.x86_64   docker://19.3.0
k8s-master3        Ready    master   3d8h   v1.17.0                                192.168.1.106   <none>        CentOS Linux 7 (Core)   3.10.0-1062.el7.x86_64   docker://19.3.0
k8s-node1          Ready    <none>   3d8h   v1.17.0                                192.168.1.108   <none>        CentOS Linux 7 (Core)   3.10.0-1062.el7.x86_64   docker://19.3.0

2、导入环境变量

2.1、导出zun集群的环境变量文件

admin-openrc.sh

2.2、在k8s-master1节点上执行

交互式执行需要输入密码，密码为123456

./admin-openrc.sh

还要增加一些环境变量，这些环境变量可以从zun的openstack集群中获取

# cat ~/keystonerc_admin
unset OS_SERVICE_TOKEN
export OS_USERNAME=admin
export OS_PASSWORD='123456'
export OS_AUTH_URL=http://192.168.166.104:5000/v3
export PS1='[\u@\h \W(keystone_admin)]\$ '
export OS_PROJECT_NAME=admin
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default
export OS_IDENTITY_API_VERSION=3

将 ~/keystonerc_admin 拷贝到k8s-master1上，执行

source keystonerc_admin

3、运行virtual-kubelet

3.1、将编译好的virtual-kubelet拷贝到k8s-master1

3.2、执行virtual-kubelet

./virtual-kubelet --provider openstack

3.3、查看k8s节点

# kubectl get nodes
NAME               STATUS   ROLES    AGE    VERSION
k8s-master1        Ready    master   3d8h   v1.17.0
k8s-master2        Ready    master   3d8h   v1.17.0
k8s-master3        Ready    master   3d8h   v1.17.0
k8s-node1          Ready    <none>   3d8h   v1.17.0
virtual-kubelet    Ready    agent    31h    v1.14.3-vk-openstack-zun-781b46a-dev

4、部署一个pod到virtual-kubelet上

4.1、编辑nginx.yaml

# cat nginx.yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  tolerations:
  - key: "virtual-kubelet.io/provider"
    operator: "Equal"
    value: "openstack"
    effect: "NoSchedule"
  containers:
  - name: nginx
    image: 192.168.166.187:5000/nginx:1.13.3
    imagePullPolicy: IfNotPresent
    ports:
    - containerPort: 80

4.2、运行nginx pod

# kubectl apply -f nginx.yaml

4.3、k8s集群查看nginx pod

nginx pod已经运行在虚拟的node上

# kubectl get pods -o wide
NAME            READY   STATUS    RESTARTS   AGE    IP              NODE               NOMINATED NODE   READINESS GATES
nginx   1/1     Running   0          165m   172.200.1.188   virtual-kubelet    <none>           <none>

4.4、zun集群查看pod

在zun中，与pod对应的概念是capsule

# zun capsule-list
+--------------------------------------+-------------------------------+---------+---------------+
| uuid                                 | name                          | status  | addresses     |
+--------------------------------------+-------------------------------+---------+---------------+
| a6715753-48a2-404f-b8e1-9413fed8dbc0 | capsule-test                  | Running | 172.200.1.111 |
| b49d5f7f-0f40-4d01-a983-35349266d4bb | default-nginx                 | Running | 172.200.1.163 |
| 2a21aaf4-dff2-42f2-9e7b-4e576174cdf4 | kube-system-kube-proxy-48cb5  | Running | 172.200.1.164 |
| 3363420a-9a7a-45b2-8fc6-a3e4070eb389 | kube-system-calico-node-dlbv7 | Running | 172.200.1.118 |
| 1da5ec1a-8f98-4eff-800d-bae4d4a78dc6 | default-sc-ips-mp-0-0         | Running | 172.200.1.188 |
| 22a78f09-67fc-49ea-942a-46ecb54ac07a | kube-system-kube-proxy-pvhrs  | Running | 172.200.1.168 |
| 30825dc7-d44d-4575-8b0d-8666c84f2a6f | kube-system-calico-node-k5vzl | Running | 172.200.1.153 |
| 03210a5f-d59c-453c-847e-b9d329e8b6c4 | default-sc-ips-mp-0-1         | Running | 172.200.1.200 |
+--------------------------------------+-------------------------------+---------+---------------+