RabbitMQ启动失败

问题原因: 由于服务器异常宕机导致RabbitMQ挂掉，服务器恢复之后尝试启动MQ发现启动失败。报错信息如下

[root@bogon rabbitMq]# rabbitmqctl start_app
Error: unable to perform an operation on node ‘rabbit@iZbp128yw4rvtfbytgv4y7Z’. Please see diagnostics information and suggestions below.

Most common reasons for this are:

• Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues)
• CLI tool fails to authenticate with the server (e.g. due to CLI tool’s Erlang cookie not matching that of the server)
• Target node is not running

In addition to the diagnostics info below:

• See the CLI, clustering and networking guides on http://rabbitmq.com/documentation.html to learn more
• Consult server logs on node rabbit@iZbp128yw4rvtfbytgv4y7Z
• If target node is configured to use long node names, don’t forget to use --longnames with CLI tools

DIAGNOSTICS

attempted to contact: [rabbit@iZbp128yw4rvtfbytgv4y7Z]

rabbit@iZbp128yw4rvtfbytgv4y7Z:

• connected to epmd (port 4369) on iZbp128yw4rvtfbytgv4y7Z
• epmd reports node ‘rabbit’ uses port 25672 for inter-node and CLI tool traffic
• can’t establish TCP connection to the target node, reason: timeout (timed out)
• suggestion: check if host ‘iZbp128yw4rvtfbytgv4y7Z’ resolves, is reachable and ports 25672, 4369 are not blocked by firewall

Current node details:

• node name: ‘rabbitmqcli-5146-rabbit@iZbp128yw4rvtfbytgv4y7Z’
• effective user’s home directory: /var/lib/rabbitmq
• Erlang cookie hash: mVWZ9hzwnH+BCsNzXPPxQA==

查看状态:

rabbitmqctl status
报错如下:

Error: unable to perform an operation on node ‘rabbit@iZbp128yw4rvtfbytgv4y7Z’. Please see diagnostics information and suggestions below.

Most common reasons for this are:

Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues)
CLI tool fails to authenticate with the server (e.g. due to CLI tool’s Erlang cookie not matching that of the server)
Target node is not running
In addition to the diagnostics info below:

See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more
Consult server logs on node rabbit@iZbp128yw4rvtfbytgv4y7Z
If target node is configured to use long node names, don’t forget to use --longnames with CLI tools
DIAGNOSTICS

attempted to contact: [‘rabbit@iZbp128yw4rvtfbytgv4y7Z’]

rabbit@iZbp128yw4rvtfbytgv4y7Z:

connected to epmd (port 4369) on iZbp128yw4rvtfbytgv4y7Z
epmd reports: node ‘rabbit’ not running at all
no other nodes on iZbp128yw4rvtfbytgv4y7Z
suggestion: start the node
Current node details:

node name: ‘rabbitmqcli-11079-rabbit@iZbp128yw4rvtfbytgv4y7Z’
effective user’s home directory: /root
Erlang cookie hash: vVAgrz18VW8gDZQB2YRW8A==

解决:

有可能是因为我们停止的时候进程并未完全关闭或者是被其他应用占用,导致后续重启失败,rabbitmq没有启动.
Rabbitmq-server默认占用的端口号有：5672、15672、25672、4369
查询lsof -i :4369端口占用进程pidkill -9 pid杀死相关进程之后重启发现报错依旧,且4369端口kill时显示进程不存在但是重新查询端口占用发现另一个进程占用该端口
Error: unable to perform an operation on node ‘rabbit@iZbp128yw4rvtfbytgv4y7Z’.根据这段错误信息查询到有可能是ip映射问题,故在 /etc/hosts 文件做一个映射，就将hostname iZbp128yw4rvtfbytgv4y7Z与本机ip映射了一下。保存，source /etc/hosts 。启动MQ报错依旧
ifconfig #查询到本机内网IP为: 10.25.0.181
#方式一:
echo 10.25.0.181 iZbp128yw4rvtfbytgv4y7Z >> /etc/host #直接将映射内容写入host文件
#方式二:
vim /etc/hosts #按 i 进入编辑模式

添加一条ip映射到本机

10.25.0.181 iZbp128yw4rvtfbytgv4y7Z

Esc 退出编辑模式

:wq #保存文件并退出
source /etc/hosts #刷新映射文件 报错-bash: 127.0.0.1: 未找到命令 -bash: ::1: 未找到命令 -bash: 10.25.0.181: 未找到命令
本人最终解决方式
journalctl -xe #查看系统日志排查MQ启动失败原因
系统日志如下:

– Subject: Unit rabbitmq-server.service has failed
– Defined-By: systemd
– Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

– Unit rabbitmq-server.service has failed.

– The result is failed.
8月 19 11:15:38 iZbp128yw4rvtfbytgv4y7Z systemd[1]: Unit rabbitmq-server.service entered failed state.
8月 19 11:15:38 iZbp128yw4rvtfbytgv4y7Z systemd[1]: rabbitmq-server.service failed.
8月 19 11:15:48 iZbp128yw4rvtfbytgv4y7Z systemd[1]: rabbitmq-server.service holdoff time over, scheduling restart.
8月 19 11:15:48 iZbp128yw4rvtfbytgv4y7Z systemd[1]: Stopped RabbitMQ broker.
– Subject: Unit rabbitmq-server.service has finished shutting down
– Defined-By: systemd
– Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

– Unit rabbitmq-server.service has finished shutting down.
8月 19 11:15:48 iZbp128yw4rvtfbytgv4y7Z systemd[1]: Starting RabbitMQ broker…
– Subject: Unit rabbitmq-server.service has begun start-up
– Defined-By: systemd
– Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

– Unit rabbitmq-server.service has begun starting up.
8月 19 11:15:49 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Configuring logger redirection
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: ## ## RabbitMQ 3.8.19
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: ## ##
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: ########## Copyright © 2007-2021 VMware, Inc. or its affiliates.
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: ###### ##
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: ########## Licensed under the MPL 2.0. Website: https://rabbitmq.com
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Erlang: 23.3.4.4 [emu]
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: TLS Library: OpenSSL - OpenSSL 1.0.2k-fips 26 Jan 2017
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Doc guides: https://rabbitmq.com/documentation.html
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Support: https://rabbitmq.com/contact.html
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Tutorials: https://rabbitmq.com/getstarted.html
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Monitoring: https://rabbitmq.com/monitoring.html
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Logs: /var/log/rabbitmq/[e-mail]
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: /var/log/rabbitmq/rabbit@iZbp128yw4rvtfbytgv4y7Z_upgrade.log
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Config file(s): (none)
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Starting broker…
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: BOOT FAILED
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: ===========
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Error during startup: {error,
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: {cannot_delete_plugins_expand_dir,
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: ["/var/lib/rabbitmq/mnesia/rabbit@iZbp128yw4rvtfbytgv4y7Z-plugins-expand",
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: {cannot_delete,
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: “/var/lib/rabbitmq/mnesia/rabbit@iZbp128yw4rvtfbytgv4y7Z-plugins-expand/cowboy-2.8.0/ebin/cowboy_app.beam”,
8月 19 11:15:53 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: eacces}]}}
8月 19 11:15:55 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: {“Kernel pid terminated”,application_controller,"{application_start_failure,rabbit,{{cannot_delete_plugins_expand_dir,["/var/lib/rabbitmq/mnesia/rabbit@iZbp128yw4rvtfb
8月 19 11:15:55 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Kernel pid terminated (application_controller) ({application_start_failure,rabbit,{{cannot_delete_plugins_expand_dir,["/var/lib/rabbitmq/mnesia/rabbit@iZbp128yw4rvtfbyt
8月 19 11:15:55 iZbp128yw4rvtfbytgv4y7Z rabbitmq-server[16624]: Crash dump is being written to: /var/log/rabbitmq/erl_crash.dump…done
8月 19 11:15:55 iZbp128yw4rvtfbytgv4y7Z systemd[1]: rabbitmq-server.service: main process exited, code=exited, status=1/FAILURE
8月 19 11:15:55 iZbp128yw4rvtfbytgv4y7Z systemd[1]: Failed to start RabbitMQ broker.

删除/var/lib/rabbitmq/mnesia 目录下的[e-mail]、rabbit@iZbp128yw4rvtfbytgv4y7Z、rabbit@iZbp128yw4rvtfbytgv4y7Z-plugins-expand后，再使用systemctl start rabbitmq-server启动，成功启动MQ。
注意 :文件中包含交换机队列及用户等信息,删除等于重置MQ(队列会被清空),请谨慎操作