rocketmq在Kubernetes(k8s)中的集群配置，2m-2s-async：多Master多Slave模式，异步复制
单于季
目标 rocketmq 集群
2m-2s-async：多Master多Slave模式，异步复制
宿主机IP: 192.168.0.254
fox.风
源码 地址 https://github.com/foxiswho/k8s-nacos-sentinel-rocketmq-zipkin-elasticsearch-redis-mysql
配置目录
mkdir -p /www/k8s/foxdev/rocketmq/logs-1/
mkdir -p /www/k8s/foxdev/rocketmq/store-1/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-1a/
mkdir -p /www/k8s/foxdev/rocketmq/store-1a/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-1as/
mkdir -p /www/k8s/foxdev/rocketmq/store-1as/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-1as2/
mkdir -p /www/k8s/foxdev/rocketmq/store-1as2/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-2/
mkdir -p /www/k8s/foxdev/rocketmq/store-2/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-2b/
mkdir -p /www/k8s/foxdev/rocketmq/store-2b/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-2bs/
mkdir -p /www/k8s/foxdev/rocketmq/store-2bs/commitlog
mkdir -p /www/k8s/foxdev/rocketmq/logs-2bs2/
mkdir -p /www/k8s/foxdev/rocketmq/store-2bs2/commitlog
chmod -R 777 /www/k8s/foxdev/rocketmq/*
logs开头目录为 日志目录
store开头目录为 数据目录
-1 表示 rocketmq nameserver
-1a 表示 rocketmq broker
-1as 表示 rocketmq broker slave
logs-ui 为 rocketmq web ui 管理 日志目录
其他以此类推
相关说明
nodePort 后面表示 外部端口
port 后面表示 内部端口
集群1端口如下
spec:
type: NodePort
ports:
- port: 9876
targetPort: 9876
nodePort: 30876
name: rmqser
- port: 30911
targetPort: 30911
nodePort: 30911
name: rmqbroker
- port: 30921
targetPort: 30921
nodePort: 30921
name: rmqbrokerslave
- port: 30931
targetPort: 30931
nodePort: 30931
name: rmqbrokerslave2
集群2端口如下
spec:
type: NodePort
ports:
- port: 9876
targetPort: 9876
nodePort: 30877
name: rmqser2
- port: 30961
targetPort: 30961
nodePort: 30961
name: rmqbroker2
- port: 30971
targetPort: 30971
nodePort: 30971
name: rmqbrokerslave2
- port: 30981
targetPort: 30981
nodePort: 30981
name: rmqbrokerslave22
web ui 端口 和 管理员与密码
端口 30879 ，管理员账户和密码如下
admin
admin
账户和密码 在 配置文件在 rocketmq/conf/users.properties 文件中修改，
# This file supports hot change, any change will be auto-reloaded without Console restarting.
# Format: a user per line, username=password[,N] #N is optional, 0 (Normal User); 1 (Admin)
# Define Admin
admin=admin,1
第一个admin 表示 用户名
第二个admin 表示 密码
1 表示 管理员 角色
集群1配置
broker 配置 broker-a.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#所属集群名字
brokerClusterName=DefaultCluster
#broker名字，注意此处不同的配置文件填写的不一样，如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-a
#0 表示Master，>0 表示Slave
brokerId=0
#nameServer地址，分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876
#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);，解决方式2 brokerIP1 设置宿主机IP，不要使用docker 内部IP
brokerIP1=192.168.0.254
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭 ！！！这里仔细看是false，false，false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=30911
#haService中使用 默认值为：listenPort + 1
#haListenPort=30912
#主要用于slave同步master listenPort - 2
#fastListenPort=30909
#删除文件时间点，默认凌晨4点
deleteWhen=04
#文件保留时间，默认48小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
broker-slave 配置 broker-a-s.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#所属集群名字
brokerClusterName=DefaultCluster
#broker名字，注意此处不同的配置文件填写的不一样，如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-a
#0 表示Master，>0 表示Slave
brokerId=1
#nameServer地址，分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876
#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);，解决方式2 brokerIP1 设置宿主机IP，不要使用docker 内部IP
brokerIP1=192.168.0.254
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭 ！！！这里仔细看是false，false，false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=30921
#haService中使用 默认值为：listenPort + 1
#haListenPort=31912
#主要用于slave同步master listenPort - 2
#fastListenPort=31909
#删除文件时间点，默认凌晨4点
deleteWhen=04
#文件保留时间，默认48小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
broker-slave 配置 broker-a-s2.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#所属集群名字
brokerClusterName=DefaultCluster
#broker名字，注意此处不同的配置文件填写的不一样，如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-a
#0 表示Master，>0 表示Slave
brokerId=2
#nameServer地址，分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876
#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);，解决方式2 brokerIP1 设置宿主机IP，不要使用docker 内部IP
brokerIP1=192.168.0.254
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭 ！！！这里仔细看是false，false，false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=30931
#haService中使用 默认值为：listenPort + 1
#haListenPort=31912
#主要用于slave同步master listenPort - 2
#fastListenPort=31909
#删除文件时间点，默认凌晨4点
deleteWhen=04
#文件保留时间，默认48小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
k8s pod
rocketmq1-pod.yml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: rocketmq
labels:
app: rocketmq
spec:
serviceName: rocketmq
replicas: 1
selector:
matchLabels:
app: rocketmq
template:
metadata:
labels:
app: rocketmq
spec:
containers:
- name: rocketmq
image: foxiswho/rocketmq:4.8.0
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 450m
memory: 2000Mi
requests:
cpu: 400m
memory: 2000Mi
env:
- name: TZ
value: Asia/Shanghai
- name: JAVA_OPT
value: -Duser.home=/home/rocketmq
- name: JAVA_OPT_EXT
value: -Xmn256m
command: ["sh","-c","mqnamesrv"]
ports:
- containerPort: 9876
volumeMounts:
- name: vol-logs
mountPath: /home/rocketmq/logs
- name: vol-store
mountPath: /home/rocketmq/store
- name: rmqbroker
image: foxiswho/rocketmq:4.8.0
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 440m
memory: 2000Mi
requests:
cpu: 400m
memory: 2000Mi
env:
- name: TZ
value: Asia/Shanghai
- name: JAVA_OPT_EXT
value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-a.conf" ]
ports:
- containerPort: 30909
- containerPort: 30911
- containerPort: 30912
volumeMounts:
- name: vol-logs-a
mountPath: /home/rocketmq/logs
- name: vol-store-a
mountPath: /home/rocketmq/store
- name: vol-conf
mountPath: /etc/rocketmq/
- name: rmqbroker-slave
image: foxiswho/rocketmq:4.8.0
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 450m
memory: 2000Mi
requests:
cpu: 400m
memory: 2000Mi
env:
- name: TZ
value: Asia/Shanghai
- name: JAVA_OPT_EXT
value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-a-s.conf" ]
ports:
- containerPort: 30919
- containerPort: 30921
- containerPort: 30922
volumeMounts:
- name: vol-logs-as
mountPath: /home/rocketmq/logs
- name: vol-store-as
mountPath: /home/rocketmq/store
- name: vol-conf
mountPath: /etc/rocketmq/
- name: rmqbroker-slave2
image: foxiswho/rocketmq:4.8.0
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 450m
memory: 2000Mi
requests:
cpu: 400m
memory: 2000Mi
env:
- name: TZ
value: Asia/Shanghai
- name: JAVA_OPT_EXT
value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-a-s2.conf" ]
ports:
- containerPort: 30929
- containerPort: 30931
- containerPort: 30932
volumeMounts:
- name: vol-logs-as2
mountPath: /home/rocketmq/logs
- name: vol-store-as2
mountPath: /home/rocketmq/store
- name: vol-conf
mountPath: /etc/rocketmq/
volumes:
- name: vol-logs
hostPath:
path: /www/k8s/foxdev/rocketmq/logs-1
type: Directory
- name: vol-store
hostPath:
path: /www/k8s/foxdev/rocketmq/store-1
type: Directory
- name: vol-conf
hostPath:
path: /www/k8s/foxdev/rocketmq/conf
type: Directory
- name: vol-logs-a
hostPath:
path: /www/k8s/foxdev/rocketmq/logs-1a
type: Directory
- name: vol-store-a
hostPath:
path: /www/k8s/foxdev/rocketmq/store-1a
type: Directory
- name: vol-logs-as
hostPath:
path: /www/k8s/foxdev/rocketmq/logs-1as
type: Directory
- name: vol-store-as
hostPath:
path: /www/k8s/foxdev/rocketmq/store-1as
type: Directory
- name: vol-logs-as2
hostPath:
path: /www/k8s/foxdev/rocketmq/logs-1as2
type: Directory
- name: vol-store-as2
hostPath:
path: /www/k8s/foxdev/rocketmq/store-1as2
type: Directory
k8s service
rocketmq1-service.yml
apiVersion: v1
kind: Service
metadata:
name: rocketmq
labels:
app: rocketmq
spec:
type: NodePort
ports:
- port: 9876
targetPort: 9876
nodePort: 30876
name: rmqser
- port: 30911
targetPort: 30911
nodePort: 30911
name: rmqbroker
- port: 30921
targetPort: 30921
nodePort: 30921
name: rmqbrokerslave
- port: 30931
targetPort: 30931
nodePort: 30931
name: rmqbrokerslave2
selector:
app: rocketmq
集群2配置
broker 配置 broker-b.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#所属集群名字
brokerClusterName=DefaultCluster
#broker名字，注意此处不同的配置文件填写的不一样，如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-b
#0 表示Master，>0 表示Slave
brokerId=0
#nameServer地址，分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876
#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);，解决方式2 brokerIP1 设置宿主机IP，不要使用docker 内部IP
brokerIP1=192.168.0.254
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭 ！！！这里仔细看是false，false，false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=30961
#haService中使用 默认值为：listenPort + 1
#haListenPort=30912
#主要用于slave同步master listenPort - 2
#fastListenPort=30909
#删除文件时间点，默认凌晨4点
deleteWhen=04
#文件保留时间，默认48小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
broker-slave 配置 broker-b-s.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#所属集群名字
brokerClusterName=DefaultCluster
#broker名字，注意此处不同的配置文件填写的不一样，如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-b
#0 表示Master，>0 表示Slave
brokerId=1
#nameServer地址，分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876
#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);，解决方式2 brokerIP1 设置宿主机IP，不要使用docker 内部IP
brokerIP1=192.168.0.254
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭 ！！！这里仔细看是false，false，false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=30971
#haService中使用 默认值为：listenPort + 1
#haListenPort=30912
#主要用于slave同步master listenPort - 2
#fastListenPort=30909
#删除文件时间点，默认凌晨4点
deleteWhen=04
#文件保留时间，默认48小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
broker-slave 配置 broker-b-s2.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#所属集群名字
brokerClusterName=DefaultCluster
#broker名字，注意此处不同的配置文件填写的不一样，如果在broker-a.properties使用:broker-a,
#在broker-b.properties使用:broker-b
brokerName=broker-b
#0 表示Master，>0 表示Slave
brokerId=2
#nameServer地址，分号分割
#namesrvAddr=rocketmq-0:9876;rocketmq-1:9876
namesrvAddr=rocketmq-0.rocketmq.default.svc.cluster.local:9876;rocketmq2-0.rocketmq2.default.svc.cluster.local:9876
#启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed
# 解决方式1 加上一句producer.setVipChannelEnabled(false);，解决方式2 brokerIP1 设置宿主机IP，不要使用docker 内部IP
brokerIP1=192.168.0.254
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭 ！！！这里仔细看是false，false，false
#原因下篇博客见~ 哈哈哈哈
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=30981
#haService中使用 默认值为：listenPort + 1
#haListenPort=30912
#主要用于slave同步master listenPort - 2
#fastListenPort=30909
#删除文件时间点，默认凌晨4点
deleteWhen=04
#文件保留时间，默认48小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/home/rocketmq/store
#commitLog 存储路径
#storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog
#消费队列存储
#storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue
#消息索引存储路径
#storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index
#checkpoint 文件存储路径
#storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint
#abort 文件存储路径
#abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
k8s pod
rocketmq2-pod.yml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: rocketmq2
labels:
app: rocketmq2
spec:
serviceName: rocketmq2
replicas: 1
selector:
matchLabels:
app: rocketmq2
template:
metadata:
labels:
app: rocketmq2
spec:
containers:
- name: rocketmq
image: foxiswho/rocketmq:4.8.0
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 450m
memory: 2000Mi
requests:
cpu: 400m
memory: 2000Mi
env:
- name: TZ
value: Asia/Shanghai
- name: JAVA_OPT
value: -Duser.home=/home/rocketmq
- name: JAVA_OPT_EXT
value: -Xmn256m
command: ["sh","-c","mqnamesrv"]
ports:
- containerPort: 9876
volumeMounts:
- name: vol-logs
mountPath: /home/rocketmq/logs
- name: vol-store
mountPath: /home/rocketmq/store
- name: rmqbroker
image: foxiswho/rocketmq:4.8.0
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 440m
memory: 2000Mi
requests:
cpu: 400m
memory: 2000Mi
env:
- name: TZ
value: Asia/Shanghai
- name: JAVA_OPT_EXT
value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-b.conf" ]
ports:
- containerPort: 30959
- containerPort: 30961
- containerPort: 30962
volumeMounts:
- name: vol-logs-a
mountPath: /home/rocketmq/logs
- name: vol-store-a
mountPath: /home/rocketmq/store
- name: vol-conf
mountPath: /etc/rocketmq/
- name: rmqbroker-slave
image: foxiswho/rocketmq:4.8.0
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 450m
memory: 2000Mi
requests:
cpu: 400m
memory: 2000Mi
env:
- name: TZ
value: Asia/Shanghai
- name: JAVA_OPT_EXT
value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-b-s.conf" ]
ports:
- containerPort: 30969
- containerPort: 30971
- containerPort: 30972
volumeMounts:
- name: vol-logs-as
mountPath: /home/rocketmq/logs
- name: vol-store-as
mountPath: /home/rocketmq/store
- name: vol-conf
mountPath: /etc/rocketmq/
- name: rmqbroker-slave2
image: foxiswho/rocketmq:4.8.0
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 450m
memory: 2000Mi
requests:
cpu: 400m
memory: 2000Mi
env:
- name: TZ
value: Asia/Shanghai
- name: JAVA_OPT_EXT
value: -Duser.home=/home/rocketmq -Xms1024M -Xmx1024M -Xmn256m
command: [ "sh","-c","mqbroker -c /etc/rocketmq/broker-b-s2.conf" ]
ports:
- containerPort: 30979
- containerPort: 30981
- containerPort: 30982
volumeMounts:
- name: vol-logs-as2
mountPath: /home/rocketmq/logs
- name: vol-store-as2
mountPath: /home/rocketmq/store
- name: vol-conf
mountPath: /etc/rocketmq/
volumes:
- name: vol-logs
hostPath:
path: /www/k8s/foxdev/rocketmq/logs-2
type: Directory
- name: vol-store
hostPath:
path: /www/k8s/foxdev/rocketmq/store-2
type: Directory
- name: vol-conf
hostPath:
path: /www/k8s/foxdev/rocketmq/conf
type: Directory
- name: vol-logs-a
hostPath:
path: /www/k8s/foxdev/rocketmq/logs-2b
type: Directory
- name: vol-store-a
hostPath:
path: /www/k8s/foxdev/rocketmq/store-2b
type: Directory
- name: vol-logs-as
hostPath:
path: /www/k8s/foxdev/rocketmq/logs-2bs
type: Directory
- name: vol-store-as
hostPath:
path: /www/k8s/foxdev/rocketmq/store-2bs
type: Directory
- name: vol-logs-as2
hostPath:
path: /www/k8s/foxdev/rocketmq/logs-2bs2
type: Directory
- name: vol-store-as2
hostPath:
path: /www/k8s/foxdev/rocketmq/store-2bs2
type: Directory
k8s service
rocketmq2-service.yml
apiVersion: v1
kind: Service
metadata:
name: rocketmq2
labels:
app: rocketmq2
spec:
type: NodePort
ports:
- port: 9876
targetPort: 9876
nodePort: 30877
name: rmqser2
- port: 30961
targetPort: 30961
nodePort: 30961
name: rmqbroker2
- port: 30971
targetPort: 30971
nodePort: 30971
name: rmqbrokerslave2
- port: 30981
targetPort: 30981
nodePort: 30981
name: rmqbrokerslave22
selector:
app: rocketmq2
UI
pod
ui-rc.yml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: rmqui
labels:
app: rmqui
spec:
serviceName: rmqui
replicas: 1
selector:
matchLabels:
app: rmqui
template:
metadata:
labels:
app: rmqui
spec:
containers:
- name: rmqui
image: pongsky/rocketmq-console-ng
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 450m
memory: 1024Mi
requests:
cpu: 400m
memory: 768Mi
env:
- name: TZ
value: Asia/Shanghai
- name: JAVA_OPTS
value: -Drocketmq.config.namesrvAddr=rocketmq:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false -Duser.home=/root -Drocketmq.config.dataPath=/tmp/rocketmq-console/data -Xms768m -Xmx768m
ports:
- containerPort: 8080
volumeMounts:
- name: vol-logs
mountPath: /root/logs
- name: vol-conf
mountPath: /tmp/rocketmq-console/data/users.properties
volumes:
- name: vol-logs
hostPath:
path: /www/k8s/foxdev/rocketmq/logs-ui
type: Directory
- name: vol-conf
hostPath:
path: /www/k8s/foxdev/rocketmq/conf/users.properties
type: File
service
ui-service.yml
apiVersion: v1
kind: Service
metadata:
name: rmqui
labels:
app: rmqui
spec:
type: NodePort
ports:
- port: 8080
targetPort: 8080
nodePort: 30879
selector:
app: rmqui
创建执行
kubectl create -f ui-rc.yml
kubectl create -f ui-service.yml
kubectl create -f rocketmq1-pod.yml
kubectl create -f rocketmq1-service.yml
kubectl create -f rocketmq2-pod.yml
kubectl create -f rocketmq2-service.yml
至此创建完成
类似资料：