使用dm-cache，将SSD用作HDD的Cache

---

2017/04/10
大致步骤，原理及其他可替代方法参考下面的blog
参考：SSD Caching Using dm-cache Tutorial

---

---

配置SSD Cache

1.查看用于Cache的SSD的实际大小

metadata的大小为

4MB+(16B * nr_blocks)

nr_blocks是cache设备的block的数量。本文使用256KB（262144B）作为cache block的大小。忽略SSD上分配给metadata的块。那么设备大小为（B）:

$ sudo blockdev --getsize64 /dev/nvme0n1p1
107374182400

2. 计算metadata占用的空间

1中得到的是字节数，将其转换为block数，并计算metadata所需分配的空间大小。

4194304 + (16 * 107374182400 / 262144)
10747904
10747904 / 512
20992

因此要给metadata分配20992个block。注意，如果计算结果不是整数，应该向上取整。

3.建立ssd-metadata dm设备并清空

$ sudo dmsetup create ssd-metadata --table '0 20992 linear /dev/nvme0n1p1 0'
$ sudo dd if=/dev/zero of=/dev/mapper/ssd-metadata

4.计算剩余的块数分配给ssd-cache

107374182400 / 512
209715200
209715200 - 20992
209694208

5.建立ssd-cache

$ sudo dmsetup create ssd-cache --table '0 209694208 linear /dev/nvme0n1p1 20992'

6.获得origin设备的扇区数

$ sudo blockdev --getsz /dev/sda2
1953513472

7.建立origin设备（256kB）

$ sudo dmsetup create hdd-origin --table '0 1953513472 cache /dev/mapper/ssd-metadata /dev/mapper/ssd-cache /dev/sda2 512 1 writeback default 0'

8.验证

$ ls -l /dev/mapper/hdd-origin
lrwxrwxrwx 1 root root 7 April  10 18:16 /dev/mapper/hdd-origin -> ../dm-2
$ sudo dmsetup status /dev/mapper/hdd-origin
0 1953513472 cache 8 817/2624 512 0/409559 0 43 0 0 0 0 0 1 writeback 2 migration_threshold 2048 smq 0 rw - 

9.挂载

$ sudo mkdir /mnt/cache
$ sudo mount /dev/mapper/hdd-origin /mnt/cache 

配置自启动

ssd-cache-wait.conf:

description "Suspend dmcache for SSD cache"
author "Kyle Manna <[e-mail]>"

# Start monitoring with inotifywait (will block) after the filesystem is
# mounted, when the filesystem is unmounted, teardown the dm devices.

start on mounted MOUNTPOINT=/mnt/cache

script
    inotifywait -e unmount $MOUNTPOINT
    dmsetup suspend hdd-origin
    dmsetup remove hdd-origin
    dmsetup remove ssd-cache
    dmsetup remove ssd-metadata
end script

ssd-cache.conf:

description "Configure dmcache for SSD cache"
author "Kyle Manna <[e-mail]>"

# This would be handy, but it assumes the source device is already
# available, which in this case it isn't.
#start on mounting MOUNTPOINT=/mnt/cache
# Run after /proc and /dev are finalized
start on virtual-filesystems

task
script
    # Setup the /dev/mapper/hdd-origin entity so that mountall will automount it after reading fstab
    dmsetup create ssd-metadata --table '0 20992 linear /dev/nvme0n1p1 0'
    dmsetup create ssd-cache --table '0 209694208 linear /dev/nvme0n1p1 20992'
    dmsetup create hdd-origin --table '0 1953513472 cache /dev/mapper/ssd-metadata /dev/mapper/ssd-cache /dev/sda2 512 1 writeback default 0'
    dmsetup resume hdd-origin
end script

把以上两个文件放到/etc/init下。

停用Cache

先清空，然后才可以停用

$ sudo umount /dev/mapper/hdd-origin
$ sudo dmsetup table hdd-origin
0 1953513472 cache 252:0 252:1 8:2 512 1 writeback default 0
$ sudo dmsetup status hdd-origin
0 1953513472 cache 8 817/2624 512 7/409559 104 46 1 0 0 7 0 1 writeback 2 migration_threshold 2048 smq 0 rw - 
$ sudo dmsetup suspend hdd-origin
$ sudo dmsetup reload hdd-origin --table '0 1953513472 cache 252:0 252:1 8:2 512 0 cleaner 0'
$ sudo dmsetup resume hdd-origin
$ sudo dmsetup wait hdd-origin
<wait for dirty data blocks to be written out>
$ dmsetup remove hdd-origin
$ dmsetup remove ssd-cache
$ dmsetup remove ssd-metadata

完成后可以使用以下命令来检测文件系统

$ sudo fsck /dev/sda2

参考文档

Device Mapper Documention
Device Mapper - Cache
Device Mapper - Cache Policies