Oracle Flex Cluster介绍及hub和leaf节点转换

童花蜂

2023-12-01

1 说明

Oracle Flex Cluster is a new Oracle Clusterware based topology utilizing two types of cluster nodes: Hub Nodes and Leaf Nodes. Hub Nodes represent traditional nodes, tightly coupled using network and storage. Leaf Nodes are a new type of node that runs a lighter weight stack and does not require direct shared storage connectivity.

Oracle Flex Cluster中重要的就是引入了leaf节点，leaf节点是不需要连接到共享存储的。而hub节点和传统的节点没有什么区别。最多支持64个Hub节点，每个Hub节点可以有更多的Leaf节点。

Leaf Node:Servers that are loosely coupled with Hub Nodes, which may not have direct access to the shared storage.

Hub Node:A node in and Oracle Flex Cluster that is tightly connected with other servers and has direct access to a shared disk.

Oracle Flex ASM介绍：https://blog.csdn.net/qianglei6077/article/details/92583009

2 实验1-Hub转换为Leaf

下面介绍一下如何在Hub和Leaf节点之间互相切换。

2.1 环境

当前都是基于12c R1 12.1.0.2。

2.2 查看当前集群的状态

[grid@rac1 ~]$ crsctl get cluster mode status
Cluster is running in "flex" mode

–当前集群节点都是Hub节点

[grid@rac1 ~]$ crsctl get node role config -all
Node 'rac1' configured role is 'hub'
Node 'rac2' configured role is 'hub'

2.3 将Hub转换为Leaf

[root@rac2 ~]# /u01/app/12.2.0/grid/bin/crsctl set node role leaf
CRS-4408: Node 'rac2' configured role successfully changed; restart Oracle High Availability Services for new role to take effect.

提示很清楚，已经转换成功，但是不支持在线转换，需要重启才能生效

2.4 重启CRS

2.4.1 关闭CRS

[root@rac2 ~]# /u01/app/12.2.0/grid/bin/crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac2'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac2'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac2'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac2'
CRS-2673: Attempting to stop 'ora.OCR_VOTING.dg' on 'rac2'
CRS-2673: Attempting to stop 'ora.cndba.db' on 'rac2'
CRS-2677: Stop of 'ora.DATA.dg' on 'rac2' succeeded
CRS-2677: Stop of 'ora.OCR_VOTING.dg' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac2'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.rac2.vip' on 'rac2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac2'
CRS-2677: Stop of 'ora.rac2.vip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.rac2.vip' on 'rac1'
CRS-2677: Stop of 'ora.cndba.db' on 'rac2' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac1'
CRS-2676: Start of 'ora.rac2.vip' on 'rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac2'
CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac2' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac1'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac2'
CRS-2677: Stop of 'ora.ons' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac2'
CRS-2677: Stop of 'ora.net1.network' on 'rac2' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac2' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac2'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac2'
CRS-2673: Attempting to stop 'ora.storage' on 'rac2'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac2'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac2'
CRS-2677: Stop of 'ora.storage' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac2'
CRS-2677: Stop of 'ora.ctssd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac2'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac2'
CRS-2677: Stop of 'ora.cssd' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'rac2'
CRS-2677: Stop of 'ora.crf' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac2'
CRS-2677: Stop of 'ora.gipcd' on 'rac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.

2.4.2 --启动CRS

[root@rac2 ~]# /u01/app/12.2.0/grid/bin/crsctl start crs -wait
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2'
CRS-2672: Attempting to start 'ora.evmd' on 'rac2'
CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded
CRS-2676: Start of 'ora.evmd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2'
CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rac2'
CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac2'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac2'
CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac2'
CRS-2672: Attempting to start 'ora.ctssd' on 'rac2'
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac2' succeeded
CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'rac2'
CRS-2676: Start of 'ora.storage' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'rac2'
CRS-2676: Start of 'ora.crf' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac2'
CRS-2676: Start of 'ora.crsd' on 'rac2' succeeded
CRS-6017: Processing resource auto-start for servers: rac2
CRS-6016: Resource auto-start has completed for server rac2
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.

2.5 查看Leaf节点集群服务状态

[grid@rac2 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       rac1                     STABLE
ora.DATA.dg
               ONLINE  ONLINE       rac1                     STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       rac1                     STABLE
ora.LISTENER_LEAF.lsnr
               OFFLINE OFFLINE      rac2                     STABLE
ora.OCR_VOTING.dg
               ONLINE  ONLINE       rac1                     STABLE
ora.net1.network
               ONLINE  ONLINE       rac1                     STABLE
ora.ons
               ONLINE  ONLINE       rac1                     STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac1                     STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rac1                     STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rac1                     STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       rac1                     169.254.21.89,STABLE
ora.asm
      1        ONLINE  ONLINE       rac1                     Started,STABLE
 2 ONLINE OFFLINE Instance Shutdown,ST ABLE
ora.cndba.db
      1        ONLINE  ONLINE       rac1                     Open,STABLE
 2 ONLINE OFFLINE Instance Shutdown,ST ABLE
ora.cvu
      1        ONLINE  ONLINE       rac1                     STABLE
ora.gns
      1        ONLINE  ONLINE       rac1                     STABLE
ora.gns.vip
      1       ONLINE  ONLINE       rac1                     STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       rac1                     Open,STABLE
ora.oc4j
      1        ONLINE  ONLINE       rac1                     STABLE
ora.rac1.vip
      1        ONLINE  ONLINE       rac1                     STABLE
ora.rac2.vip
      1        ONLINE  INTERMEDIATE rac1                     FAILED OVER,STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       rac1                     STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       rac1                     STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       rac1                     STABLE
--------------------------------------------------------------------------------
[grid@rac2 ~]$ srvctl status asm
ASM is running on rac1

[root@rac2 ~]# ps -ef|grep pmon
root      8509 21156  0 11:46 pts/5    00:00:00 grep --color=auto pmon

可以看到12cR1中leaf节点的ASM和数据库实例都是关闭的。12cR2中数据库实例可以以只读方式运行。

2.6 更新inventory

注意：12cR1中还需要手动更新inventory，12cR2中已不再需要

在所有Hub节点执行

[grid@rac1 ~]$ /u01/app/12.2.0/grid/oui/bin/runInstaller -updateNodeList ORACLE_HOME=/u01/app/oracle/12.2.0/db_1 "CLUSTER_NODES={rac1}" -silent -local CRS=TRUE

在新的Leaf节点执行：

[grid@rac1 ~]$ /u01/app/12.2.0/grid/oui/bin/runInstaller -updateNodeList ORACLE_HOME=/u01/app/oracle/12.2.0/db_1 "CLUSTER_NODES={rac2}" -silent -local CRS=TRUE

2.7 查看集群节点角色

[root@rac2 ~]# /u01/app/12.2.0/grid/bin/crsctl get node role config -all

Node 'rac1' configured role is 'hub'

Node 'rac2' configured role is 'leaf'

3 实验2-Leaf转换为Hub

实际上步骤都相同，操作也一样。

在Leaf节点执行：/u01/app/12.2.0/grid/bin/crsctl set node role hub
关闭CRS
打开CRS
更新inventory

4 总结

转换节点角色，需要重启CRS
12cR1中还需要手动更新inventory，12cR2中已不再需要
12cR1中leaf节点的ASM和数据库实例都是关闭的。12cR2中数据库实例可以以只读方式运行。

那这个leaf节点在12c R1中是啥作用，竟然无法运行数据库实例。对这个问题的介绍，从国外的一个博客上似乎找到了一些答案：http://www.orafaq.com/node/2857