java - 为什么 hbase 无法运行在 docker 中？

Question

我需要让 hbase 跑在 arm 上

而且想用 docker

所以我写了下面的 Dockerfile

FROM python:3.10-buster RUN . /etc/os-release && cat > /etc/apt/sources.list <<EOF    deb http://mirrors.tuna.tsinghua.edu.cn/debian/ ${VERSION_CODENAME} main contrib non-free    deb http://mirrors.tuna.tsinghua.edu.cn/debian/ ${VERSION_CODENAME}-updates main contrib non-free    deb http://mirrors.tuna.tsinghua.edu.cn/debian/ ${VERSION_CODENAME}-backports main contrib non-free    deb http://mirrors.tuna.tsinghua.edu.cn/debian-security buster/updates main contrib non-freeEOFRUN apt-get updateRUN apt-get install -y vimRUN apt-get install -y openjdk-11-jdkRUN apt-get install -y wgetWORKDIR /rootRUN wget -P /root https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/2.4.17/hbase-2.4.17-src.tar.gzRUN tar xvf /root/hbase-2.4.17-src.tar.gz

我先跑在 x86 上，看看 ok 不 ok ，所以上面的 java home 是 amd64 的

但是不行

╰─➤  docker run --rm -it ponponon/hbase-arm bash               root@97bc49430241:~# cd /root/hbase-2.4.17root@97bc49430241:~/hbase-2.4.17# root@97bc49430241:~/hbase-2.4.17# root@97bc49430241:~/hbase-2.4.17# HBASE_HOME=/root/hbase-2.4.17 JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar bash bin/hbase-daemon.sh start master running master, logging to /root/hbase-2.4.17/logs/hbase--master-97bc49430241.out/root/hbase-2.4.17/bin/hbase-daemon.sh: line 92: kill: (29) - No such processCaused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.master.HMaster

在我的虚拟机上，直接 JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 bash bin/hbase-daemon.sh start master 就行了！

我的虚拟机是 ubuntu18.04，可以直接 sudo apt install openjdk-8-jdk，所以有 jdk8，但是用的 debian10 已经没有 debian8 了，我就用 jdk11 了

https://hbase.apache.org/book.html#java

不知道是不是因为 jdk11 引起的

---

应该不是 jdk 的问题，因为我换成 ubuntu20 的镜像之后，安装 jdk8 之后也是不行

FROM ubuntu:focalRUN . /etc/os-release && cat > /etc/apt/sources.list <<EOFdeb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal main restricted universe multiversedeb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-updates main restricted universe multiversedeb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-backports main restricted universe multiversedeb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-security main restricted universe multiverseEOFRUN apt-get updateRUN apt-get install -y vimRUN apt-get install -y openjdk-8-jdkRUN apt-get install -y wgetWORKDIR /rootRUN wget -P /root https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/2.4.17/hbase-2.4.17-src.tar.gzRUN tar xvf /root/hbase-2.4.17-src.tar.gzRUN echo "export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-arm64" | tee -a /etc/profile >/dev/null

注意，这是 arm 版本

docker run --rm -it ponponon/hbase-arm-focal bash

root@d36c7ca4a53d:~/hbase-2.4.17# JAVA_HOME=/usr/lib/jvm/java-8-openjdk-arm64 bash bin/hbase-daemon.sh start masterrunning master, logging to /root/hbase-2.4.17/bin/../logs/hbase--master-d36c7ca4a53d.out/root/hbase-2.4.17/bin/hbase-daemon.sh: line 92: kill: (267) - No such process

---

不是用 harisekhon/hbase 是因为在 apple arm mac 上会报错 TTransportException: TTransportException(type=4, message='TSocket read 0 bytes')

version: "3"services:  hbase:    container_name: hbase    image: harisekhon/hbase:2.1    restart: always    ports:      - "2181:2181"      - "8080:8080"      - "8085:8085"      - "9090:9090"      - "9095:9095"      - "16000:16000"      - "16010:16010"      - "16201:16201"      - "16020:16020"      - "16030:16030"      - "16301:16301"

启动容器

─➤  docker-compose up -d                             [+] Building 0.0s (0/0)                                                                                                                           [+] Running 1/0 ⠿ Container hbase                                                                                                                                [+] Running 1/2.1s  ⠿ Container hbase                                                                                                                                [+] Running 1/2.2s  ⠿ Container hbase                                                                                                                                [+] Running 2/2.3s  ✔ Container hbase                                                                                                                                      Started0.3s  ! hbase The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s 

使用下面的 python 脚本连接

import happybase# 定义HBase连接信息host = '192.168.38.191'  # HBase主机名或IP地址host = '127.0.0.1'  # HBase主机名或IP地址port = 9090  # HBase端口，默认为9090# 建立HBase连接connection = happybase.Connection(host=host, port=port)# 获取所有表名table_names = connection.tables()# 删除所有表print('删除所有表')for table_name in table_names:    connection.delete_table(table_name, disable=True)table_name = 'my_table'  # 表名column_families = {    'cf1': dict(),  # 第一个列族    'cf2': dict()   # 第二个列族}# 建立HBase连接connection = happybase.Connection(host=host, port=port)# 创建表print('创建表')connection.create_table(table_name, column_families)# 获取表对象table = connection.table(table_name)# 插入数据print('插入数据')row_key = 'row1'data = {    'cf1:column1': 'value1',    'cf1:column2': 'value2',    'cf2:column3': 'value3'}table.put(row_key, data)# 获取数据row = table.row(row_key)print(row)# 扫描数据print('读取数据')scan_result = table.scan(row_start=row_key, row_stop=row_key)for key, value in scan_result:    print(key, value)# 删除数据table.delete(row_key)# 关闭连接connection.close()

会报错

─➤  python -u "/Users/ponponon/Desktop/code/me/ideaboom/5.py"Traceback (most recent call last):  File "/Users/ponponon/Desktop/code/me/ideaboom/5.py", line 14, in <module>    table_names = connection.tables()  File "/Users/ponponon/.local/share/virtualenvs/ideaboom-B0dr_aXc/lib/python3.10/site-packages/happybase/connection.py", line 242, in tables    names = self.client.getTableNames()  File "/Users/ponponon/.local/share/virtualenvs/ideaboom-B0dr_aXc/lib/python3.10/site-packages/thriftpy2/thrift.py", line 219, in _req    return self._recv(_api)  File "/Users/ponponon/.local/share/virtualenvs/ideaboom-B0dr_aXc/lib/python3.10/site-packages/thriftpy2/thrift.py", line 231, in _recv    fname, mtype, rseqid = self._iprot.read_message_begin()  File "thriftpy2/protocol/cybin/cybin.pyx", line 463, in cybin.TCyBinaryProtocol.read_message_begin  File "thriftpy2/protocol/cybin/cybin.pyx", line 68, in cybin.read_i32  File "thriftpy2/transport/buffered/cybuffered.pyx", line 65, in thriftpy2.transport.buffered.cybuffered.TCyBufferedTransport.c_read  File "thriftpy2/transport/buffered/cybuffered.pyx", line 69, in thriftpy2.transport.buffered.cybuffered.TCyBufferedTransport.read_trans  File "thriftpy2/transport/cybase.pyx", line 61, in thriftpy2.transport.cybase.TCyBuffer.read_trans  File "/Users/ponponon/.local/share/virtualenvs/ideaboom-B0dr_aXc/lib/python3.10/site-packages/thriftpy2/transport/socket.py", line 131, in read    raise TTransportException(type=TTransportException.END_OF_FILE,thriftpy2.transport.base.TTransportException: TTransportException(type=4, message='TSocket read 0 bytes')

Answer 1

解决了，可看: 借助 Docker 在 Arm 设备上运行 Hbase

Answer 2

你直接使用hbase的基础镜像不就行了

docker run -d -h 127.0.0.1 -p 2181:2181 -p 8080:8080 -p 8085:8085 -p 9090:9090 -p 9000:9000  -p 9095:9095  -p 16000:16000  -p 16010:16010  -p 16201:16201  -p 16301:16301 -p 16020:16020 --name hbase  harisekhon/hbase

如果想自己改造就
Dockerfile

FROM harisekhon/hbasexxxxxxx.......