Ceph高级工具介绍之ceph-objectstore-tool的使用
杜起运
0 简介
ceph-objectstore-tool工具是Ceph提供的一个low level（可以操作object和pg）的工具，可以用来查看、修改、删除ObjectStore上的object、pg以及dump OSD journal中的数据，请谨慎使用这个工具，它有可能会造成数据的永久丢失，可能这也是很少有文章介绍它的一个原因吧。
1 使用
ceph-objectstore-tool目前没有manpage，它的help文档写的也不是很友好。
[root@ceph03 ~]# ceph-objectstore-tool -h
Must provide --data-path
Allowed options:
--help produce help message
--type arg Arg is one of [filestore (default), memstore]
--data-path arg path to object store, mandatory
--journal-path arg path to journal, mandatory for filestore type
--pgid arg PG id, mandatory for info, log, remove, export,
rm-past-intervals, mark-complete
--op arg Arg is one of [info, log, remove, mkfs, fsck,
fuse, export, import, list, fix-lost, list-pgs,
rm-past-intervals, dump-journal, dump-super,
meta-list, get-osdmap, set-osdmap,
get-inc-osdmap, set-inc-osdmap, mark-complete]
--epoch arg epoch# for get-osdmap and get-inc-osdmap, the
current epoch in use if not specified
--file arg path of file to export, import, get-osdmap,
set-osdmap, get-inc-osdmap or set-inc-osdmap
--mountpoint arg fuse mountpoint
--format arg (=json-pretty) Output format which may be json, json-pretty,
xml, xml-pretty
--debug Enable diagnostic output to stderr
--force Ignore some types of errors and proceed with
operation - USE WITH CAUTION: CORRUPTION POSSIBLE
NOW OR IN THE FUTURE
--skip-journal-replay Disable journal replay
--skip-mount-omap Disable mounting of omap
--head Find head/snapdir when searching for objects by
name
--dry-run Don't modify the objectstore
Positional syntax:
ceph-objectstore-tool ... <object> (get|set)-bytes [file]
ceph-objectstore-tool ... <object> set-(attr|omap) <key> [file]
ceph-objectstore-tool ... <object> (get|rm)-(attr|omap) <key>
ceph-objectstore-tool ... <object> get-omaphdr
ceph-objectstore-tool ... <object> set-omaphdr [file]
ceph-objectstore-tool ... <object> list-attrs
ceph-objectstore-tool ... <object> list-omap
ceph-objectstore-tool ... <object> remove
ceph-objectstore-tool ... <object> dump
ceph-objectstore-tool ... <object> set-size
ceph-objectstore-tool ... <object> remove-clone-metadata <cloneid>
<object> can be a JSON object description as displayed by --op list.
<object> can be an object name which will be looked up in all the OSD's PGs.
<object> can be the empty string ('') which with a provided pgid specifies the pgmeta object
The optional [file] argument will read stdin or write stdout if not specified or if '-' specified.ceph-objectstore-tool --data-path <osd的data目录的路径> --journal-path <osd journal的路径> --type <objectstore的类型> --op <具体操作的类型> ，其中type目前支持filestore和bluestore两种，op操作，支持help种列举出来的操作。
3 使用事例
下面介绍一下，ceph-objectstore-tool的常用用法
1）list osd上有哪些pg，比如如果要list osd.2种存储哪些pg，可以执行以下命令：
<span style="font-size:10px;">[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path
/var/lib/ceph/osd/ceph-2/journal --type filestore --op list-pgs</span>2）除了能list pg之外，还能list某个pg的pg_info信息
[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path
/var/lib/ceph/osd/ceph-2/journal --type filestore --pgid 0.0 --op info</span></span>
[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --type filestore --pgid 0.38 --op list
HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
["0.38",{"oid":"rbd_id.test","key":"","snapid":-2,"hash":2586801272,"max":0,"pool":0,"namespace":"","max":0}]
["0.38",{"oid":"rbd_data.6e742ae8944a.0000000000000000","key":"","snapid":-2,"hash":3591771128,"max":0,"pool":0,"namespace":"","max":0}]3) dump某个pg的pg_log信息
[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --type filestore --pgid 0.36 --op log
HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
{
"pg_log_t": {
"head": "400'27",
"tail": "0'0",
"log": [
{
"op": "modify ",
"object": "0:6c8c8ef5:::rb.0.5e9a.2ae8944a.0000000000db:head",
"version": "376'1",
"prior_version": "0'0",
"reqid": "client.24221.1:135326",
"extra_reqids": [],
"mtime": "2016-07-21 22:19:29.377000",
"mod_desc": {
"object_mod_desc": {
"can_local_rollback": false,
"rollback_info_completed": true,
"ops": []
}
}
},
...
]
},
"pg_missing_t": {
"missing": []
},
"map": {
"divergent_priors": []
}4) remove某个pg
[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --type filestore --pgid 0.36 --op remove5) export/import 一个pg
[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --type filestore --pgid 0.36 --op export --file 1.txt
[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --type filestore --op import --file 1.txt6) 该命令还可以dump出OSD的super_block信息
[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal --op dump-super
HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
{
"cluster_fsid": "7b4eb697-fd53-4bd8-9b7a-ca1b37110623",
"osd_fsid": "3549e257-373a-44a0-a7d9-99068e9854dc",
"whoami": 2,
"current_epoch": 442,
"oldest_map": 1,
"newest_map": 442,
"weight": 0.000000,
"compat": {
"compat": {},
"ro_compat": {},
"incompat": {
"feature_1": "initial feature set(~v.18)",
"feature_2": "pginfo object",
"feature_3": "object locator",
"feature_4": "last_epoch_clean",
"feature_5": "categories",
"feature_6": "hobjectpool",
"feature_7": "biginfo",
"feature_8": "leveldbinfo",
"feature_9": "leveldblog",
"feature_10": "snapmapper",
"feature_12": "transaction hints",
"feature_13": "pg meta object"
}
},
"clean_thru": 442,
"last_epoch_mounted": 439
}7）dump一个object所有的元数据信息
[root@ceph03 0.13_head]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2/ --journal-path /var/lib/ceph/osd/ceph-2/journal rbd_data.6e742ae8944a.0000000000000013 dump
HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
{
"id": {
"oid": "rbd_data.6e742ae8944a.0000000000000013",
"key": "",
"snapid": -2,
"hash": 4028149687,
"max": 0,
"pool": 0,
"namespace": "",
"max": 0
},
"info": {
"oid": {
"oid": "rbd_data.6e742ae8944a.0000000000000013",
"key": "",
"snapid": -2,
"hash": 4028149687,
"max": 0,
"pool": 0,
"namespace": ""
},
"version": "442'1060",
"prior_version": "442'1059",
"last_reqid": "osd.1.0:59339",
"user_version": 1059,
"size": 4194304,
"mtime": "2016-07-22 14:58:47.086000",
"local_mtime": "2016-07-22 14:58:47.306385",
"lost": 0,
"flags": 52,
"snaps": [],
"truncate_seq": 0,
"truncate_size": 0,
"data_digest": 3408368108,
"omap_digest": 4294967295,
"watchers": {}
},
"stat": {
"size": 4194304,
"blksize": 4096,
"blocks": 8192,
"nlink": 1
},
"SnapSet": {
"snap_context": {
"seq": 0,
"snaps": []
},
"head_exists": 1,
"clones": []
}
}8) 当然，它可以使用get-osdmap导出集群中leveldb数据库中的数据，比如OSDMap，而且可以dump出各个epoch的map，这个大家可以在下面自己试验一下。
类似资料：