barman备份软件离线安装渡劫文
卢志强
一、前言
Barman是postgreSQL数据库的一款比较好用的备份软件，目前网上很多安装部署都是以Online的模式进行安装部署的，其相关的package也可以被自动的安装部署，所以相对简单。但是，在离线模式下，安装却有很多"坑"，今天总结的这篇文章，可以帮助离线安装的小伙伴们渡劫。
二、环境描述
基础环境：redhat-7.5 x86_64
软件版本：barman-2.12
官方在说明文档描述，部署barman-2.12的时候对环境有以下要求：
• Linux/Unix
• Python >= 3.4
• Python modules:
– argcomplete
– argh >= 0.21.2
– psycopg2 >= 2.4.2
– python-dateutil
– setuptools
- six
• PostgreSQL >= 8.3
• rsync >= 3.0.4 (optional for PostgreSQL >= 9.2)
三、实际部署安装
在安装中，我使用的是当前最新版本：python3.9，以及其他python模块的最新版本，具体如下：
[root@barman soft]# ls -l
total 53404
-rw-r--r--. 1 root root 38214 Nov 23 15:32 argcomplete-1.12.1-py2.py3-none-any.whl
-rw-r--r--. 1 root root 30872 Nov 23 15:35 argh-0.26.2-py2.py3-none-any.whl
-rw-r--r--. 1 root root 1262415 Nov 13 16:48 barman-2.12.tar.gz
-rw-r--r--. 1 root root 22000 Nov 24 17:01 _bz2.cpython-36m-x86_64-linux-gnu.so
-rw-r--r--. 1 root root 25915738 Jun 25 2019 postgresql-11.4.tar.gz
-rw-r--r--. 1 root root 427474 Nov 23 23:24 psycopg2-2.8.6.tar.gz
drwxr-xr-x. 18 1000 1000 4096 Nov 24 21:59 Python-3.9.0
-rw-r--r--. 1 root root 26724009 Nov 17 11:56 Python-3.9.0.tgz
-rw-r--r--. 1 root root 227183 Nov 23 15:38 python_dateutil-2.8.1-py2.py3-none-any.whl
-rw-r--r--. 1 root root 10938 Nov 23 22:23 six-1.14.0-py2.py3-none-any.whl
1）安装常用的包和工具
yum -y install binutils gcc gcc-c++ glibc glibc-common glibc-devel ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel make sysstat openssh-clients compat-libcap1 xorg-x11-utils xorg-x11-xauth elfutils unixODBC unixODBC-devel libXp elfutils-libelf elfutils-libelf-devel smartmontools
2）安装rsync
yum install rsync -y
rpm -qa rsync
rsync-3.1.2-4.el7.x86_64
3）安装python3
tar -xvzf Python-3.8.6.tgz
cd Python-3.8.6
./configure
make && make install
rm -rf /usr/bin/python
ln -s /usr/bin/python /usr/local/bin/python3
或者：
./configure --prefix /python3
make && make install
rm -rf /usr/bin/python
rm -rf /usr/bin/pip
ln -s /python3/bin/python3 /usr/bin/python
ln -s /python3/bin/pip3 /usr/bin/pip
vi /etc/profile
export PATH=$PATH:/python3/bin
注：
需要注意的是，在redhat- 7.5下，yum的执行是通过python2.7来解释的的，由于我们用软连接变更了/usr/bin/python的定义，所以我们可能会无法使用yum：
[root@barman Python-3.9.0]# yum list
File "/usr/bin/yum", line 30
except KeyboardInterrupt, e:
^
SyntaxError: invalid syntax
此时，我们只需要在/usr/bin/yum命令中改变解释器为2.7即可：
#!/usr/bin/python2.7
import sys
try:
import yum
except ImportError:
print >> sys.stderr, """\
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:
%s
Please install a package which provides this module, or
verify that the module is installed correctly.
It's possible that the above module doesn't match the
current version of Python, which is:
%s
If you cannot solve this problem yourself, please go to
the yum faq at:
http://yum.baseurl.org/wiki/Faq
""" % (sys.exc_value, sys.version)
sys.exit(1)
sys.path.insert(0, '/usr/share/yum-cli')
try:
import yummain
yummain.user_main(sys.argv[1:], exit_code=True)
except KeyboardInterrupt, e:
print >> sys.stderr, "\n\nExiting on user cancel."
sys.exit(1)
python安装好后，会默认安装好pip和setuptools，所以不用另外安装，但是python的其他控件需要另外安装:
4）安装argcomplete：
[root@barman soft]# pip install argcomplete-1.12.1-py2.py3-none-any.whl
5）安装argh：
[root@barman soft]# pip install argh-0.26.2-py2.py3-none-any.whl
6）安装six:
[root@barman soft]# pip install six-1.14.0-py2.py3-none-any.whl
7）安装python-dateutil:
[root@barman soft]# pip install python_dateutil-2.8.1-py2.py3-none-any.whl
注：在安装python-dateutil前必须先安装six模块，否则会报错：
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Processing ./python_dateutil-2.8.1-py2.py3-none-any.whl
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/six/
8）安装psycopg2-2.8.6
[root@barman soft]# tar -xvzf psycopg2-2.8.6.tar.gz
[root@barman psycopg2-2.8.6]# cd psycopg2-2.8.6
[root@barman psycopg2-2.8.6]# python setup.py build
可以发现报了以下错误：
Error: pg_config executable not found.
pg_config is required to build psycopg2 from source. Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.
For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>).
这是因为没有找到pg_config文件，而它是在postgresql的服务器端或者客户端中所包含的，所以我们要先安装postgresql的服务端或客户端。
* 安装postgreSQL客户端：
mkdir -p /pg_client
tar -xvzf postgresql-11.4.tar.gz
yum install -y readline*
yum install -y zlib*
yum install -y gcc*
yum install -y perl*
yum install -y tcl*
cd postgresql-11.4
./configure --prefix=/pg_client --with-python --with-perl
make
make world
make -C src/bin install
make -C src/include install
make -C src/interfaces install
make -C doc install
vi /etc/profile
MANPATH=/pg_client/share/man:$MANPATH
LD_LIBRARY_PATH=/pg_client/lib
PG_HOME=/pg_client
PATH=$PATH:$HOME/bin:$PG_HOME/bin
export PATH PG_HOME LD_LIBRARY_PATH MANPATH
再次编译安装psycopg2-2.8.6
[root@barman psycopg2-2.8.6]# cd psycopg2-2.8.6
[root@barman psycopg2-2.8.6]# python setup.py build
[root@barman psycopg2-2.8.6]# python setup.py install
9）查看安装的python包：
[root@barman psycopg2-2.8.6]# pip list
Package Version
--------------- -------
argcomplete 1.12.1
argh 0.26.2
pip 20.2.1
psycopg2 2.8.6
python-dateutil 2.8.1
setuptools 49.2.1
six 1.14.0
其实，可以看到，部署barman需要的相关Python模块有以上这些部分（有些是模块本身需要依赖的模块）。
10）安装barman
tar -xvzf barman-2.12.tar.gz
chown -R postgres:postgres barman-2.12
cd barman-2.12
python setup.py build
python setup.py install
* 验证barman:
[root@barman barman-2.12]# barman -h
发现报了如下错误：
Traceback (most recent call last):
File "/python3/bin/barman", line 33, in <module>
sys.exit(load_entry_point('barman==2.12', 'console_scripts', 'barman')())
File "/python3/bin/barman", line 25, in importlib_load_entry_point
return next(matches).load()
File "/python3/lib/python3.9/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/python3/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 790, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/python3/lib/python3.9/site-packages/barman-2.12-py3.9.egg/barman/cli.py", line 32, in <module>
import barman.config
File "/python3/lib/python3.9/site-packages/barman-2.12-py3.9.egg/barman/config.py", line 32, in <module>
from barman import output
File "/python3/lib/python3.9/site-packages/barman-2.12-py3.9.egg/barman/output.py", line 30, in <module>
from barman.infofile import BackupInfo
File "/python3/lib/python3.9/site-packages/barman-2.12-py3.9.egg/barman/infofile.py", line 27, in <module>
import barman.compression
File "/python3/lib/python3.9/site-packages/barman-2.12-py3.9.egg/barman/compression.py", line 22, in <module>
import bz2
File "/python3/lib/python3.9/bz2.py", line 18, in <module>
from _bz2 import BZ2Compressor, BZ2Decompressor
ModuleNotFoundError: No module named '_bz2'
这是由于python缺少了_bz2.cpython-36m-x86_64-linux-gnu.so文件，解决该问题的处理方式是：
下载这个文件，或者在python3.6的安装版本中可以找到，其路径位置是：/python3/lib/python3.6/lib-dynload，也可以直接从网上下载：
链接：https://pan.baidu.com/s/1rPd7KSe6rpxxVVW2Bga-Vg
提取码：yptk
然后改名，并复制到/python3/lib/python3.9/lib-dynload下
需要注意：
这里改名需要注意，在网上查到的方式是将_bz2.cpython-36m-x86_64-linux-gnu.so中的36m改成对应版本的规则，比如python3.7，就改成_bz2.cpython-36m-x86_64-linux-gnu.so
但是在python 3.8和python3.9中，改成38m和39m是无效的；
我们最好是到/python3/lib/python3.9/lib-dynload目录下，看一下其他文件命名规则：
......
-rwxr-xr-x. 1 root root 65768 Nov 24 21:58 _testmultiphase.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root 1198000 Nov 24 21:58 unicodedata.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root 27288 Nov 24 21:59 xxlimited.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root 144472 Nov 24 21:58 _xxsubinterpreters.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root 66104 Nov 24 21:58 _xxtestfuzz.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root 160336 Nov 24 21:58 zlib.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root 135440 Nov 24 21:58 _zoneinfo.cpython-39-x86_64-linux-gnu.so
......
可以发现这里的版本都不是39m了，直接是39
所以对于该文件，我们要改名为：_bz2.cpython-39-x86_64-linux-gnu.so
[root@barman lib-dynload]# cp /home/soft/_bz2.cpython-36m-x86_64-linux-gnu.so ./_bz2.cpython-39-x86_64-linux-gnu.so
加执行权限：
[root@barman lib-dynload]# chmod +x _bz2.cpython-39-x86_64-linux-gnu.so
再次执行barman，会发现报如下错误：
ImportError: libbz2.so.1.0: cannot open shared object file: No such file or directory
* 安装bzip2
[root@barman lib-dynload]# yum install -y bzip2*
[root@barman lib-dynload]# cd /usr/lib64
[root@barman lib64]# ls -l libbz2.so.1.0.6
-rwxr-xr-x. 1 root root 68192 Jul 31 2014 libbz2.so.1.0.6
创建连接：
[root@barman lib64]# ln -s libbz2.so.1.0.6 libbz2.so.1.0
再次测试验证：
[root@barman lib64]# barman -h
usage: barman [-h] [-v] [-c CONFIG] [--color {never,always,auto}] [--log-level {NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}] [-q] [-d] [-f {console,json}]
{archive-wal,backup,check,check-backup,cron,delete,diagnose,get-wal,list-backup,list-files,list-server,put-wal,rebuild-xlogdb,receive-wal,recover,show-backup,show-server,replication-status,status,switch-wal,switch-xlog,sync-info,sync-backup,sync-wals}
...
positional arguments:
{archive-wal,backup,check,check-backup,cron,delete,diagnose,get-wal,list-backup,list-files,list-server,put-wal,rebuild-xlogdb,receive-wal,recover,show-backup,show-server,replication-status,status,switch-wal,switch-xlog,sync-info,sync-backup,sync-wals}
archive-wal Execute maintenance operations on WAL files for a given server. This command processes any incoming WAL files for the server and archives
them along the catalogue.
backup Perform a full backup for the given server (supports 'all')
check Check if the server configuration is working. This command returns success if every checks pass, or failure if any of these fails
check-backup Make sure that all the required WAL files to check the consistency of a physical backup (that is, from the beginning to the end of the full
backup) are correctly archived. This command is automatically invoked by the cron command and at the end of every backup operation.
cron Run maintenance tasks (global command)
delete Delete a backup
diagnose Diagnostic command (for support and problems detection purpose)
get-wal Retrieve WAL_NAME file from SERVER_NAME archive. The content will be streamed on standard output unless the --output-directory option is
specified.
list-backup List available backups for the given server (supports 'all')
list-files List all the files for a single backup
list-server List available servers, with useful information
put-wal Receive a WAL file from SERVER_NAME and securely store it in the incoming directory. The file will be read from standard input in tar format.
rebuild-xlogdb Rebuild the WAL file database guessing it from the disk content.
receive-wal Start a receive-wal process. The process uses the streaming protocol to receive WAL files from the PostgreSQL server.
recover Recover a server at a given time, name, LSN or xid
show-backup This method shows a single backup information
show-server Show all configuration parameters for the specified servers
replication-status Shows live information and status of any streaming client
status Shows live information and status of the PostgreSQL server
switch-wal Execute the switch-wal command on the target server
switch-xlog Execute the switch-wal command on the target server
sync-info Output the internal synchronisation status. Used to sync_backup with a passive node
sync-backup Command that synchronises a backup from a master to a passive node
sync-wals Command that synchronises WAL files from a master to a passive node
optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-c CONFIG, --config CONFIG
uses a configuration file (defaults: ~/.barman.conf, /etc/barman.conf, /etc/barman/barman.conf)
--color {never,always,auto}, --colour {never,always,auto}
Whether to use colors in the output (default: 'auto')
--log-level {NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}
Override the default log level
-q, --quiet be quiet (default: False)
-d, --debug debug output (default: False)
-f {console,json}, --format {console,json}
output format (default: 'console')
Barman by 2ndQuadrant (www.2ndQuadrant.com)
可以发现barman可以正常运行，安装成功！
11）复制配置文件
将配置复制到/etc下，安装全部完成：
cp -r /home/soft/barman-2.12/doc/barman.conf /etc/barman.conf
cp -r /home/soft/barman-2.12/doc/barman.d /etc/barman.d
类似资料：