【DRBD 8】Install
龚浩宕
Install DRBD (Distributed Replicated Block Device) to configure Distributed Storage System.
This example is based on the environment like follows.
+----------------------+ | +----------------------+
| [ DRBD Server#1 ] |10.0.0.50 | 10.0.0.60| [ DRBD Server#2 ] |
| node01.srv.world +----------+----------+ node02.srv.world |
| | | |
+----------------------+ +----------------------+
It’s necessary the server you’d like to install DRBD has free block-device.
This example shows to configure to use a device [/dev/vg_r0/lv_r0].
Basic Environment Preparation
[root@node01 ~]# hostnamectl set-hostname node01.srv.world
[root@node01 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 19G 0 part
├─centos-root 253:0 0 17G 0 lvm /
└─centos-swap 253:1 0 2G 0 lvm [SWAP]
sdb 8:16 0 1G 0 disk
sr0 11:0 1 1024M 0 rom
# create lvm (Each node needs to execute)
[root@node01 ~]# pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created.
[root@node01 ~]# vgcreate vg_r0 /dev/sdb
Volume group "vg_r0" successfully created
[root@node01 ~]# lvcreate -n lv_r0 -l 100%VG vg_r0
Logical volume "lv_r0" created.
[root@node01 ~]# vgs
VG #PV #LV #SN Attr VSize VFree
centos 1 2 0 wz--n- <19.00g 0
vg_r0 1 1 0 wz--n- 1020.00m 0
Install
[1] Update system and install required packages first and reboot on all Nodes.
[root@node01 ~]# yum -y update
[root@node01 ~]# yum -y install gcc gcc-c++ make automake autoconf libxslt libxslt-devel flex rpm-build kernel-devel
[root@node01 ~]# reboot
[2] Install DRBD on all Nodes.
Download DRBD 8 from the official site below.
⇒ https://www.linbit.com/en/drbd-community/drbd-download/
By the way, drbd-km package is built with the current version of kernel, so if you will update kernel in the future, then you need to re-build DRBD again with the version of kernel.
[root@node01 ~]# mkdir -p rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
[root@node01 ~]# wget pkg.linbit.com/downloads/drbd/8.4/drbd-8.4.11-1.tar.gz \
pkg.linbit.com/downloads/drbd/utils/drbd-utils-9.19.1.tar.gz
[root@node01 ~]# tar -xf drbd-8.4.11-1.tar.gz
[root@node01 ~]# cd drbd-8.4.11-1
[root@node01 drbd-8.4.11-1]# make km-rpm
[root@node01 drbd-8.4.11-1]# cd ../
[root@node01 ~]# tar zxvf drbd-utils-9.5.0.tar.gz
[root@node01 ~]# cd drbd-utils-9.5.0
[root@node01 drbd-utils-9.5.0]# vi drbd.spec.in
# line 34: add
%undefine with_sbinsymlinks
[root@node01 drbd-utils-9.5.0]# ./configure
[root@node01 drbd-utils-9.5.0]# make rpm
[root@node01 drbd-utils-9.5.0]# cd /root/rpmbuild/RPMS/x86_64
[root@data1 x86_64]# rpm -Uvh drbd-km-3.10.0_1160.76.1.el7.x86_64-8.4.11-1.x86_64.rpm \
drbd-utils-9.5.0-1.el7.x86_64.rpm \
drbd-bash-completion-9.5.0-1.el7.x86_64.rpm
Preparing... ################################# [100%]
Updating / installing...
1:drbd-utils-9.5.0-1.el7 ################################# [ 33%]
2:drbd-km-3.10.0_1160.76.1.el7.x86_################################# [ 67%]
3:drbd-bash-completion-9.5.0-1.el7 ################################# [100%]
[3] Install DRBD by rpm pkg
[root@node01 ~]# rpm -ivh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm
[root@node01 ~]# yum install -y drbd84-utils kmod-drbd84
[4] If Firewalld is running, allow service ports.
[root@node01 ~]# firewall-cmd --add-port=6996-7800/tcp --permanent
success
[root@node01 ~]# firewall-cmd --reload
success
DRBD 8 : Configure
[1] Configure like follows on all Nodes.
[root@node01 ~]# vi /etc/drbd.d/global_common.conf
# add follows in the disk section (detach disk if IO errors happen)
disk {
on-io-error detach;
[root@node01 ~]# vi /etc/drbd.d/r0.res
# create new
resource r0 {
# DRBD device
device /dev/drbd0;
# block device
disk /dev/vg_r0/lv_r0;
meta-disk internal;
on node01.srv.world {
# IP-address:port
address 10.0.0.50:7788;
}
on node02.srv.world {
address 10.0.0.60:7788;
}
}
# load module
[root@node01 ~]# echo drbd > /etc/modules-load.d/drbd.conf
[root@node01 ~]# modprobe drbd
[root@node01 ~]# lsmod | grep drbd
drbd 397041 0
libcrc32c 12644 2 xfs,drbd
# create DRBD resource
[root@node01 ~]# drbdadm create-md r0
initializing activity log
initializing bitmap (32 KB) to all zero
Writing meta data...
New drbd meta data block successfully created.
[root@node01 ~]# systemctl enable drbd
[root@node01 ~]# systemctl start drbd
[2] After configuring on all Nodes, Sync data on a Node.
# current status is "Secondary/Secondary"
[root@node01 ~]# cat /proc/drbd
version: 8.4.11-1 (api:1/proto:86-101)
GIT-hash: 66145a308421e9c124ec391a7848ac20203bb03c build by root@node01.srv.world, 2022-10-20 23:23:09
0: cs:Connected ro:Secondary/Secondary ds:Diskless/Inconsistent C r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
# get primary role and sync data
[root@node01 ~]# drbdadm -- --overwrite-data-of-peer primary r0
# sync starts
[root@node01 ~]# cat /proc/drbd
version: 8.4.11-1 (api:1/proto:86-101)
GIT-hash: 66145a308421e9c124ec391a7848ac20203bb03c build by root@node01.srv.world, 2018-07-12 19:28:32
0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
ns:1047352 nr:0 dw:0 dr:1049480 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:82832036
[>....................] sync'ed: 1.3% (80888/81912)M
finish: 0:56:34 speed: 24,392 (15,868) K/sec
# after syncing, the status turns like follows
[root@node01 ~]# cat /proc/drbd
version: 8.4.11-1 (api:1/proto:86-101)
GIT-hash: 66145a308421e9c124ec391a7848ac20203bb03c build by root@node01.srv.world, 2018-07-12 19:28:32
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:83879388 nr:0 dw:0 dr:83881516 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[3] It’s OK to configure DRBD, create file system on DRBD device and mount it to use.
[root@node01 ~]# mkfs.xfs /dev/drbd0
[root@node01 ~]# mkdir /drbd_disk
[root@node01 ~]# mount /dev/drbd0 /drbd_disk
[root@node01 ~]# df -hT
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/cl-root xfs 26G 1.7G 25G 7% /
devtmpfs devtmpfs 2.0G 0 2.0G 0% /dev
tmpfs tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs tmpfs 2.0G 8.5M 2.0G 1% /run
tmpfs tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/vda1 xfs 1014M 238M 777M 24% /boot
tmpfs tmpfs 396M 0 396M 0% /run/user/0
/dev/drbd0 xfs 80G 33M 80G 1% /drbd_disk
# create a test file
[root@node01 ~]# echo 'test file' > /drbd_disk/test.txt
[root@node01 ~]# ll /drbd_disk
total 4
-rw-r--r--. 1 root root 10 Jul 12 19:52 test.txt
[4] To mount DRBD device on the secondary Host, do it like follows.
########### on primary Node ###########
# unmount and get secondary role
[root@node01 ~]# umount /drbd_disk
[root@node01 ~]# drbdadm secondary r0
########### on secondary Node ###########
# get primary role and mount
[root@node02 ~]# drbdadm primary r0
[root@node02 ~]# mount /dev/drbd0 /drbd_disk
[root@node02 ~]# df -hT
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/cl-root xfs 26G 1.6G 25G 6% /
devtmpfs devtmpfs 2.0G 0 2.0G 0% /dev
tmpfs tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs tmpfs 2.0G 8.5M 2.0G 1% /run
tmpfs tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/vda1 xfs 1014M 238M 777M 24% /boot
tmpfs tmpfs 396M 0 396M 0% /run/user/0
/dev/drbd0 xfs 1G 33M 1G 1% /drbd_disk
[root@node02 ~]# ll /drbd_disk
total 4
-rw-r--r--. 1 root root 10 Jul 12 19:52 test.txt
类似资料：