Storm-1.0.1+ZooKeeper-3.4.8+Netty-4.1.3 HA集群安装
下载Storm-1.0.1
http://mirrors.tuna.tsinghua.edu.cn/apache/storm/apache-storm-1.0.1/apache-storm-1.0.1.tar.gz
下载ZooKeeper-3.4.8 http://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.4.8/zookeeper-3.4.8.tar.gz 下载Netty-4.1.3.Final.tar.bz2 http://akamai.bintray.com/e7/e769383d99e244b7173b496f90f9b4a00574f8a33cdec2fe40b54f957a373e01?__gda__=exp=1469008981~hmac=7f3864e58af529209d0c0460cc3d5a73f267d7b550ad82bf14c0f645f8608e74&response-content-disposition=attachment%3Bfilename%3D%22netty-4.1.3.Final.tar.bz2%22&response-content-type=application%2Fx-bzip2 软件包如下： apache-storm-1.0.1.tar.gz jdk-7u65-linux-x64.tar.gz netty-4.1.3.Final.tar.bz2 zookeeper-3.4.8.tar.gz 集群规划 主机 IP Soft Process nimbus01 192.168.14.162 Netty、Storm nimbus、core、logviewer nimbus02 192.168.14.156 Netty、Storm nimbus、logviewer supervisor01 192.168.14.59 Netty、ZooKeeper、Storm supervisor、logviewer supervisor02 192.168.14.119 Netty、ZooKeeper、Storm supervisor、logviewer supervisor03 192.168.14.139 Netty、ZooKeeper、Storm supervisor、logviewer 一、配置免密码登录 二、安装Jdk 在所有节点上安装jdk cd /usr/local/soft/ tar -zxvf jdk-7u65-linux-x64.tar.gz cd jdk1.7.0_65 vim /etc/profile export JAVA_HOME=/usr/local/soft/jdk1.7.0_65 export PATH=.:$JAVA_HOME/bin:$PATH source /etc/profile 三、安装ZooKeeper-3.4.8 在supervisor01节点上配置 cd /usr/local/soft/ tar -zxvf zookeeper-3.4.8.tar.gz cd zookeeper-3.4.8 mv conf/zoo_sample.cfg conf/zoo.cfg vim conf/zoo.cfg tickTime=2000 initLimit=10 syncLimit=5 dataDir=/usr/local/soft/zookeeper-3.4.8/data clientPort=2181 server.1=suspervisor01:2888:3888 server.2=suspervisor02:2888:3888 server.3=suspervisor03:2888:3888 mkdir data && echo "1">data/myid 复制supervisor01节点上的ZooKeeper到supervisor02上 scp -r /usr/local/soft/zookeeper-3.4.8 root@supervisor02:/usr/local/soft/ echo "2" >/usr/local/soft/zookeeper-3.4.8/data/myid 复制supervisor01节点上的ZooKeeper到supervisor03上 scp -r /usr/local/soft/zookeeper-3.4.8 root@supervisor03:/usr/local/soft/ echo "3" >/usr/local/soft/zookeeper-3.4.8/data/myid 四、安装Storm-1.0.1 在nimbus01节点上配置 cd /usr/local/soft/ tar -zxvf apache-storm-1.0.1.tar.gz cd apache-storm-1.0.1 vim conf/storm.yaml # #### Storm dependents The ZooKeeper Cluster #### storm.zookeeper.servers: - "supervisor01" - "supervisor02" - "supervisor03" # #### Storm Nimbus Nodes HA #### nimbus.seeds: ["nimbus01", "nimbus02"] # #### Storm Local Storage #### storm.local.dir: "/usr/local/soft/apache-storm-1.0.1/localdir" # #### Storm Supervisor Nodes Worker Process #### supervisor.slots.ports: - 6700 - 6701 - 6702 - 6703 # #### Not Selector ZeroMQ, Storm Use Netty Transport Protocol #### storm.messaging.transport: "org.apache.storm.messaging.netty.Context" storm.messaging.netty.server_worker_threads: 1 storm.messaging.netty.client_worker_threads: 1 storm.messaging.netty.buffer_size: 5242880 storm.messaging.netty.max_retries: 300 storm.messaging.netty.max_wait_ms: 1000 storm.messaging.netty.min_wait_ms: 100 mkdir localdir 复制nimbus01节点上的Storm到nimbus02、supervisor01、supervisor02、supervisor03上 scp -r /usr/local/soft/apache-storm-1.0.1 root@nimbus02:/usr/local/soft/ scp -r /usr/local/soft/apache-storm-1.0.1 root@supervisor01:/usr/local/soft/ scp -r /usr/local/soft/apache-storm-1.0.1 root@supervisor02:/usr/local/soft/ scp -r /usr/local/soft/apache-storm-1.0.1 root@supervisor03:/usr/local/soft/ 五、启动集群 1、ZooKeeper 启动集群 for i in supervisor01 supervisor02 supervisor03; do echo $i; ssh $i "source /etc/profile; /usr/local/soft/zookeeper-3.4.8/bin/zkServer.sh start"; done 关闭集群 for i in supervisor01 supervisor02 supervisor03; do echo $i; ssh $i "source /etc/profile; /usr/local/soft/zookeeper-3.4.8/bin/zkServer.sh stop"; done 查看ZooKeeper进程状态 for i in supervisor01 supervisor02 supervisor03; do echo $i; ssh $i "source /etc/profile; /usr/local/soft/zookeeper-3.4.8/bin/zkServer.sh status"; done 查看进程 for i in supervisor01 supervisor02 supervisor03; do echo $i; ssh $i "source /etc/profile; jps"; done 2、启动Storm（需要确保ZooKeeper集群已经启动） 启动nimbus（storm.yaml中nimbus.seeds: []的节点都需要启动nimbus进程） 在nimbus01上启动如下进程 nohup /usr/local/soft/apache-storm-1.0.1/bin/storm nimbus & nohup /usr/local/soft/apache-storm-1.0.1/bin/storm ui & nohup /usr/local/soft/apache-storm-1.0.1/bin/storm logviewer & 在nimbus02上启动如下进程 nohup /usr/local/soft/apache-storm-1.0.1/bin/storm nimbus & nohup /usr/local/soft/apache-storm-1.0.1/bin/storm logviewer & 在supervisor01上启动如下进程 nohup /usr/local/soft/apache-storm-1.0.1/bin/storm supervisor & nohup /usr/local/soft/apache-storm-1.0.1/bin/storm logviewer & 在supervisor02上启动如下进程 nohup /usr/local/soft/apache-storm-1.0.1/bin/storm supervisor & nohup /usr/local/soft/apache-storm-1.0.1/bin/storm logviewer & 在supervisor03上启动如下进程 nohup /usr/local/soft/apache-storm-1.0.1/bin/storm supervisor & nohup /usr/local/soft/apache-storm-1.0.1/bin/storm logviewer &
