AFD在CentOS环境中部署

方璞

2023-12-01

在第一节中（http://www.htz.pw/?p=1207）介绍了AFD的功能和常见的问题，其中提到AFD是内核模块，需要匹配的系统版本和内核版本匹配才能安装AFD，并且也提到Oracle未认证CentOS平台。其实大伙都知道CentOS跟Oracle Linux一样，都是基于RHEL重新打包的，系统版本和内核版本都是兼容的，那为什么Oracle认证了RHEL，就没有认证CentOS？可想而知，这跟系统版本和内核版本无任何关系，应该是Oracle从技术层面做了限制。今天我们就在CentOS平台上面测试AFD的安装和配置。

1，操作系统安装和配置

OS：CentOS 7.9
Oracle GRID：19.17（目前最新的版本）

2，软件安装

这里跳过操作系统安装和配置、Oracle集群软件安装和配置，有兴趣的可以去www.htz.pw里面搜相关的文档。

3，升级GRID软件

本环境是将软件升级到最新的版本，也可根据自己环境选择软件版本。

[root@node1 ~]# su - grid -c "\$ORACLE_HOME/gridSetup.sh -applyRU /tmp/34449117/34416665"
ERROR: Unable to verify the graphical display setup. This application requires X display. Make sure that xdpyinfo exist under PATH variable.
Preparing the home to patch...
Applying the patch /tmp/34449117/34416665...

Successfully applied the patch.
The log can be found at: /u01/app/oraInventory/logs/GridSetupActions2023-01-02_10-13-28PM/installerPatchActions_2023-01-02_10-13-28PM.log

No X11 DISPLAY variable was set, but this program performed an operation which requires it.

4，AFD配置

4.1 兼容性测试

[root@node1 ~]# export ORACLE_HOME=/oracle/app/19.3.0/grid_1
[root@node1 ~]# export ORACLE_BASE=/tmp
[root@node1 ~]# export PATH=$PATH:$ORACLE_HOME/bin
[root@node1 ~]#
[root@node1 ~]# afddriverstate supported
AFD-620: AFD is not supported on this operating system version: 'centos-release-7-9.2009.0.el7.centos.x86_64'
AFD-9201: Not Supported
AFD-9294: updating file /etc/sysconfig/oracledrivers.conf

AFD-9201: Not Supported表示Centos 7.9不支持。

修改AFD部署脚本

通过下面AFD的部署脚本，可以知道Oracle是通过rpm -qf /etc/redhat-release的方式来获取系统的厂商名字和内核版本。

[root@node1 ~]# vi $ORACLE_HOME/lib/osds_acfslib.pm

  # see - http://www.oracle.com/us/technologies/027626.pdf
  if (-e "/etc/oracle-release")
  {
    open (RPM_QF, "rpm -qf /etc/oracle-release 2>&1 |");
    $release = <RPM_QF>;
    close (RPM_QF);
  }
  elsif (-e "/etc/redhat-release")
  {
    open (RPM_QF, "rpm -qf /etc/redhat-release 2>&1 |");
    $release = <RPM_QF>;
    close (RPM_QF);
  }
  elsif (-e "/etc/os-release")
  {
    open (RPM_QF, "rpm -qf /etc/os-release 2>&1 |");
    $release = <RPM_QF>;
    close (RPM_QF);
  }
  elsif (-e "/etc/SuSE-release")
  {
    open (RPM_QF, "rpm -qf /etc/SuSE-release 2>&1 |");
    $release = <RPM_QF>;
    close (RPM_QF);
  }

[root@node1 ~]# rpm -qf /etc/redhat-release
centos-release-7-9.2009.0.el7.centos.x86_64

继续向下查看脚本，可以看到Oracle没有多Centos进行判断，只对Oracle Linux和RHEL进行判断，下面通过手动添加CentOS的判断，命令如下所示

[root@node1 ~]# sed -i '/(\$release =~ \/\^enterprise-release\/) ||/a \         (\$release =~ \/^centos-release\/) ||        # Centos Enterprise Linux' $ORACLE_HOME/lib/osds_acfslib.pm
[root@node1 ~]# sed -i 's/Centos/Redhat/' /etc/redhat-release

详细的命令代码如下：

    579   if ((defined($release)) &&                     # Redhat or OEL if defined
    580       (($release =~ /^redhat-release/) ||        # straight RH
    581        ($release =~ /^enterprise-release/) ||    # Oracle Enterprise Linux
    582        ($release =~ /^centos-release/) ||        # Oracle Linux
    583        ($release =~ /^oraclelinux-release/)))    # Oracle Linux

   1168   elsif (($release =~ /^redhat-release/) ||        # straight RH
   1169          ($release =~ /^enterprise-release/) ||    # Oracle Enterprise Linux
   1170          ($release =~ /^centos-release/) ||        # Centos Enterprise Linux
   1171          ($release =~ /^oraclelinux-release/))     # Oracle Linux
   1172   {

通过修改代码后，再次执行supported，提示已经支持。

[root@node1 ~]# afddriverstate supported
AFD-9200: Supported

4.2 配置磁盘的标签

在未安装AFD的环境中配置磁盘标签需要添加–init参数，如图所示，此步只需要再RAC环境中执行。

[root@node1 ~]# asmcmd afd_label OCR01 /dev/sdb --init
[root@node1 ~]# ls -l /dev/oracleafd/disks/*
-rw-rw-r-- 1 grid oinstall 31 Jan  3 01:23 /dev/oracleafd/disks/OCR01

5，配置单机GRID环境

如果是在集群环境中，需要再root用户下临时修改磁盘的权限，否则配置OCR磁盘组时无法查询到磁盘，修改的命令如下：

[root@node1 ~]# chown grid:asmadmin /dev/sdb

配置单机GRID环境

$ORACLE_HOME/gridSetup.sh -silent -skipPrereqs -waitForCompletion  -printtime  \
INVENTORY_LOCATION=/u01/app/oraInventory \
SELECTED_LANGUAGES=en \
oracle.install.option=HA_CONFIG \
ORACLE_BASE=/oracle/app/grid \
ORACLE_HOME=/oracle/app/19.3.0/grid_1 \
oracle.install.asm.OSDBA=asmdba \
oracle.install.asm.OSOPER=oinstall \
oracle.install.asm.OSASM=asmadmin \
oracle.install.asm.configureAFD=true \
oracle.install.asm.SYSASMPassword=Htzoracle123 \
oracle.install.asm.diskGroup.diskDiscoveryString=/dev/sd* \
oracle.install.asm.diskGroup.name=ha_ocr \
oracle.install.asm.diskGroup.disksWithFailureGroupNames=/dev/sdb, \
oracle.install.asm.diskGroup.disks=/dev/sdb \
oracle.install.asm.diskGroup.redundancy=EXTERNAL \
oracle.install.asm.diskGroup.AUSize=4 \
oracle.install.asm.monitorPassword=Htzoracle123


As a root user, execute the following script(s):
    1. /oracle/app/19.3.0/grid_1/root.sh

Execute /oracle/app/19.3.0/grid_1/root.sh on the following nodes:
[node1]



Successfully Setup Software.
As install user, execute the following command to complete the configuration.
    /oracle/app/19.3.0/grid_1/gridSetup.sh -executeConfigTools -responseFile /oracle/app/19.3.0/grid_1/install/response/grid_2023-01-03_01-30-58AM.rsp [-silent]
Note: The required passwords need to be included in the response file.


[root@node1 ~]# /oracle/app/19.3.0/grid_1/root.sh
Check /oracle/app/19.3.0/grid_1/install/root_node1_2023-01-03_01-33-28-805656094.log for the output of root script

[grid@node1 ~]$ /oracle/app/19.3.0/grid_1/gridSetup.sh -executeConfigTools -responseFile /oracle/app/19.3.0/grid_1/install/response/grid_2023-01-03_01-30-58AM.rsp -silent
Launching Oracle Grid Infrastructure Setup Wizard...

You can find the logs of this session at:
/u01/app/oraInventory/logs/GridSetupActions2023-01-03_01-38-08AM

You can find the log of this install session at:
 /u01/app/oraInventory/logs/UpdateNodeList2023-01-03_01-38-08AM.log
[WARNING] [INS-43080] Some of the configuration assistants failed, were cancelled or skipped.
   ACTION: Refer to the logs or contact Oracle Support Services.

6，确认AFD是否安装

6.1 查询OKS是否已经加载

如果oks已经加载，说明AFD已经安装成功。

[root@node1 ~]# lsmod |grep oracle
oracleacfs           5238775  0
oracleadvm           1176594  0
oracleoks             781410  2 oracleacfs,oracleadvm
oracleafd             222652  0

6.2 查看AFD磁盘标签

[grid@node1 ~]$ asmcmd afd_lsdsk
--------------------------------------------------------------------------------
Label                     Filtering   Path
================================================================================
OCR01                       ENABLED   /dev/sdb

6.3 利用AFD磁盘创建磁盘组

[grid@node1 ~]$ srvctl add asm
[grid@node1 ~]$ srvctl start asm

[grid@node1 ~]$ asmcmd afd_dsget
AFD discovery string: /dev/sd*
[grid@node1 ~]$ asmcmd dsget
parameter:AFD:*
profile:++no-value-at-resource-creation--never-updated-through-ASM++
[grid@node1 ~]$ asmca -silent -createDiskGroup -diskString AFD:* -diskGroupName ha_ocr -redundancy EXTERNAL -diskList AFD:OCR01

[DBT-30001] Disk groups created successfully. Check /oracle/app/grid/cfgtoollogs/asmca/asmca-230103AM014513.log for details.

[grid@node1 ~]$ asmcmd lsdg
State    Type    Rebal  Sector  Logical_Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512             512   4096  1048576      3072     3008                0            3008              0             N  HA_OCR/

[grid@node1 ~]$ srvctl start diskgroup -diskgroup ha_ocr
[grid@node1 ~]$ asmcmd lsdg
State    Type    Rebal  Sector  Logical_Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512             512   4096  1048576      3072     3008                0            3008              0             N  HA_OCR/

6.4 AFD标签写入信息

下面新增加一块磁盘/dev/sdb，查看AFD标签写入内容

[grid@node1 ~]$ ls -l /dev/sdb*
brw-rw---- 1 root disk 8, 16 Jan  3  2023 /dev/sdb
[grid@node1 ~]$ ls -l /dev/oracleafd/disks/*
-rw-r--r-- 1 grid asmadmin 9 Jan  3  2023 /dev/oracleafd/disks/OCR01

[root@node1 ~]# od -x /dev/sdc
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
^C

[grid@node1 ~]$ asmcmd afd_label data01 /dev/sdc
No devices to be labeled.
ASMCMD-9513: ASM disk label set operation failed.

[root@node1 ~]# export ORACLE_HOME=/oracle/app/19.3.0/grid_1
[root@node1 ~]# export ORACLE_BASE=/oracle/app/grid
[root@node1 ~]# /oracle/app/19.3.0/grid_1/bin/asmcmd afd_label data01 /dev/sdc
[root@node1 ~]# /oracle/app/19.3.0/grid_1/bin/asmcmd afd_lsdsk
--------------------------------------------------------------------------------
Label                     Filtering   Path
================================================================================
DATA01                      ENABLED   /dev/sdc
OCR01                       ENABLED   /dev/sdb

[root@node1 ~]# od -cx /dev/sdc
0000000  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0 334 261 375 221
           0000    0000    0000    0000    0000    0000    b1dc    91fd
0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
           0000    0000    0000    0000    0000    0000    0000    0000
0000040   O   R   C   L   D   I   S   K   D   A   T   A   0   1  \0  \0
           524f    4c43    4944    4b53    4144    4154    3130    0000
0000060  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
           0000    0000    0000    0000    0000    0000    0000    0000
*
0000440  \0  \0  \0  \0  \0  \0  \0  \0  \n  \n  \n 264 251 320 263   c
           0000    0000    0000    0000    0a0a    b40a    d0a9    63b3
0000460  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
           0000    0000    0000    0000    0000    0000    0000    0000
*

ORCLDISKDATA01为AFD写入磁盘头中的内容，如果AFD标签丢失时，我们也可以手动写入相应的内容。