搭建 Zookeeper+Storm 集群
1.Set up a Zookeeper cluster
2.Install dependencies on Nimbus and worker machines
3.Download and extract a Storm release to Nimbus and worker machines
4.Fill in mandatory configurations into storm.yaml
5.Launch daemons under supervision using "storm" script and a supervisor of your choice
服务器信息
ubuntu-s-storm-1 137.184.63.172 Zookeeper+Storm
ubuntu-s-storm-2 167.172.138.180 Zookeeper+Storm
ubuntu-s-storm-3 143.244.165.32 Zookeeper+Storm
一、配置 Zookeeper 集群
官方地址：https://zookeeper.apache.org/releases.html
1.下载安装apache-zookeeper
cd /usr/local/src
wget https://dlcdn.apache.org/zookeeper/zookeeper-3.7.1/apache-zookeeper-3.7.1-bin.tar.gz
2.解压
tar -zxvf apache-zookeeper-3.7.1-bin.tar.gz
mv apache-zookeeper-3.7.1-bin ../zookeeper
3. 安装 JDK9
tar -zxvf jdk-8u333-linux-x64.tar.gz && mv jdk1.8.0_333 ../jdk_1.8
cat >> /etc/profile << EOF
export JAVA_HOME=/usr/local/jdk_1.8/
export JRE_HOME=$JAVA_HOME/jre
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:./
EOF
source /etc/profile
4.创建 data and logs
cd /usr/local/zookeeper && mkdir data logs
5.更改 hosts
cat >> /etc/hosts << EOF
137.184.63.172 ubuntu-s-storm-1
167.172.138.180 ubuntu-s-storm-2
143.244.165.32 ubuntu-s-storm-3
EOF
6.配置zoo.cfg 文件
cd /usr/local/zookeeper/conf && cp zoo_sample.cfg zoo.cfg
sed -i 's/dataDir=.*/dataDir=\/usr\/local\/zookeeper\/data/g' /usr/local/zookeeper/conf/zoo.cfg
cat >> /usr/local/zookeeper/conf/zoo.cfg << EOF
dataLogDir=/usr/local/zookeeper/logs
server.1=137.184.63.172:2881:3881
server.2=167.172.138.180:2881:3881
server.3=143.244.165.32:2881:3881
EOF
说明：服务器。服务器ID=服务器IP地址：服务器间的通信端口：服务器间投票端口
客户端和服务器之间通信的默认端口：2181
服务器间通信的默认端口：2881
服务器间投票的默认端口：3881
7.更改zookeeper的默认端口，以避免与storm发生冲突
echo "admin.serverPort=8800" >> /usr/local/zookeeper/conf/zoo.cfg
8.记录每个服务器的ID，单独执行
ubuntu-s-storm-1
echo 1 >/usr/local/zookeeper/data/myid
ubuntu-s-storm-2
echo 2 >/usr/local/zookeeper/data/myid
ubuntu-s-storm-3
echo 3 >/usr/local/zookeeper/data/myid
8.启动 zookeeper
/usr/local/zookeeper/bin/zkServer.sh start/status/restart/stop
二.在Nimbus和辅助计算机上安装依赖项
1.Java 7+ (Apache Storm 1.x is tested through travis ci against both java 7 and java 8 JDKs)，安装java
2.Python 2.6.6 (Python 3.x should work too, but is not tested as part of our CI enviornment) 安装python
apt install python-is-python3
三.下载Storm版本并将其解压缩到Nimbus和worker机器
1.安装 Storm
wget https://dlcdn.apache.org/storm/apache-storm-2.4.0/apache-storm-2.4.0.tar.gz
tar -zxvf apache-storm-2.4.0.tar.gz
mv apache-storm-2.4.0 ../storm
四.在storm.yaml中填写强制配置
1.配置 storm.yaml
cat >> storm.yaml << EOF
storm.zookeeper.servers:
- "ubuntu-s-storm-1"
- "ubuntu-s-storm-2"
- "ubuntu-s-storm-3"
storm.local.dir: "/usr/local/storm/data"
nimbus.seeds: ["ubuntu-s-storm-1", "ubuntu-s-storm-2", "ubuntu-s-storm-2"]
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
EOF
2.启动 Storm
nohup /usr/local/storm/bin/storm nimbus &
nohup /usr/local/storm/bin/storm supervisor &
nohup /usr/local/storm/bin/storm ui &