第三部分：Ceph 进阶 - 10. 查看使用 RBD 镜像的客户端
优质
小牛编辑
138浏览
2023-12-01
有时候删除 rbd image 会提示当前 rbd image 正在使用中，无法删除：
rbd rm foo2016-11-09 20:16:14.018332 7f81877a07c0 -1 librbd: image has watchers - not removingRemoving image: 0% complete...failed.rbd: error: image still has watchersThis means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout.
所以希望能查看到底是谁在使用 rbd image。
对于 rbd image 的使用，目前主要有两种形式：内核模块 map 后再 mount ；通过 libvirt 等供给虚拟机使用。都是利用 rados listwatchers 去查看，只是两种形式下需要读取的文件不一样。
内核模块 map
查看方法如下：
# 查看 rbd pool 中的 imageroot@ceph1:~# rbd lsfooltesttest# 查看 foo 的使用者root@ceph1:~# rados -p rbd listwatchers foo.rbdwatcher=10.202.0.82:0/1760414582 client.1332795 cookie=1# 去对应主机上检查root@ceph2:~#rbd showmapped |grep foo0 rbd foo - /dev/rbd0
虚拟机通过 libvirt 使用 rbd image
查看方法如下：
# 查看该虚拟机卷的信息root@ceph1:~# rbd info volumes/volume-ee0c4077-a607-4bc9-a8cf-e893837361f3rbd image 'volume-ee0c4077-a607-4bc9-a8cf-e893837361f3':size 1024 MB in 256 objectsorder 22 (4096 kB objects)block_name_prefix: rbd_data.13c3277850f538format: 2features: layering, stripingflags:parent: images/3601459f-060b-460f-b73b-db74237d922e@snapoverlap: 40162 kBstripe unit: 4096 kBstripe count: 1# 查看该 image 的使用者root@ceph1:~# rados -p volumes listwatchers rbd_header.13c3277850f538watcher=10.202.0.21:0/1043073 client.1298745 cookie=140256077850496
我们登录控制节点，查看对应的 cinder 卷：
root@controller:~# cinder list | grep ee0c4077-a607-4bc9-a8cf-e893837361f3| ee0c4077-a607-4bc9-a8cf-e893837361f3 | in-use | | 1 | - | true | 96ee1aad-af27-4c9d-968f-291dbb2766a1 |
该卷挂载在 ID 为 96ee1aad-af27-4c9d-968f-291dbb2766a1 的虚拟机上。通过 nova show 命令验证该虚拟机是否在物理机 10.202.0.21 ( computer21 )上：
root@controller:~# nova show 96ee1aad-af27-4c9d-968f-291dbb2766a1+--------------------------------------+---------------------------------------------------------------------------------+| Property | Value |+--------------------------------------+---------------------------------------------------------------------------------+| OS-DCF:diskConfig | AUTO || OS-EXT-AZ:availability_zone | az_ip || OS-EXT-SRV-ATTR:host | computer21 || OS-EXT-SRV-ATTR:hostname | byp-volume || OS-EXT-SRV-ATTR:hypervisor_hostname | computer21 || OS-EXT-SRV-ATTR:instance_name | instance-00000989 || OS-EXT-SRV-ATTR:kernel_id | || OS-EXT-SRV-ATTR:launch_index | 0 || OS-EXT-SRV-ATTR:ramdisk_id | || OS-EXT-SRV-ATTR:reservation_id | r-hwiyx15c || OS-EXT-SRV-ATTR:root_device_name | /dev/vda || OS-EXT-SRV-ATTR:user_data | - || OS-EXT-STS:power_state | 1 || OS-EXT-STS:task_state | - || OS-EXT-STS:vm_state | active || OS-SRV-USG:launched_at | 2016-11-09T08:19:41.000000 || OS-SRV-USG:terminated_at | - || accessIPv4 | || accessIPv6 | || blue-net network | 192.168.50.27 || config_drive | || created | 2016-11-09T07:53:59Z || description | byp-volume || flavor | m1.small (2) || hostId | 5104ee1a0538048d6ef80b14563a0cbac461f86523c5c81f5d18069e || host_status | UP || id | 96ee1aad-af27-4c9d-968f-291dbb2766a1 || image | Attempt to boot from volume - no image supplied || key_name | octavia_ssh_key || locked | False || metadata | {} || name | byp-volume || os-extended-volumes:volumes_attached | [{"id": "ee0c4077-a607-4bc9-a8cf-e893837361f3", "delete_on_termination": true}] || progress | 0 || security_groups | default || status | ACTIVE || tenant_id | f21a9c86d7114bf99c711f4874d80474 || updated | 2016-11-09T08:19:41Z || user_id | 142d8663efce464c89811c63e45bd82e || zq-test network | 192.168.6.6 |+--------------------------------------+---------------------------------------------------------------------------------+