【中间件2】在虚拟机搭建JStrom

一 安装步骤

二 搭建Zookeeper集群

2.1 ZooKeeper 单机安装与配置

2.1.1 下载zookeeper安装包

2.1.2 解压zookeeper安装包

wu_being@JStorm-VirtualBox:~$ sudo tar -zxvf zookeeper-3.4.5.tar.gz
wu_being@JStorm-VirtualBox:~$ sudo chown -R wu_being:wu_being zookeeper-3.4.5

2.1.3 设置环境变量

JAVA_HOME=/home/wu_being/jdk1.7.0
ZOOKEEPER_HOME=/home/wu_being/zookeeper-3.4.5
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$ZOOKEEPER_HOME/lib
PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$PATH

export ZOOKEEPER_HOME
export JAVA_HOME
export CLASSPATH
export PATH

wu_being@JStorm-VirtualBox:~$source /home/wu_being/.bashrc

wu_being@JStorm-VirtualBox:~$source /etc/profile

2.1.4 配置zoo.cfg

cp zoo_sample.cfg zoo.cfg

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes. #####这是我修改后的
dataDir=/home/wu_being/zookeeper-3.4.5/data
# the port at which the clients will connect
clientPort=2181
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance #
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

2.1.5 启动zookeeper

wu_being@JStorm-VirtualBox:~/ netstat -at|grep 2181 #查看zookeeper端口
wu_being@JStorm-VirtualBox:~/ netstat -nat #查看端口信息
wu_being@JStorm-VirtualBox:~/zookeeper-3.4.5/bin$ ./zkServer.sh start #启动
wu_being@JStorm-VirtualBox:~/zookeeper-3.4.5/bin$ jps #查看启动的服务名称
wu_being@JStorm-VirtualBox:~/zookeeper-3.4.5/bin$ ./zkServer.sh stop #关闭

netstat -at|grep 2181

三 搭建JStorm集群

3.1 安装Python 2.6

-s $HOME/.pythonbrew/etc/bashrc && source $HOME/.pythonbrew/etc/bashrc
pythonbrew install 2.6.7
pythonbrew switch 2.6.7

3.2 安装JDK （1.7 版本）

3.3 安装JStorm

unzip jstorm-2.1.1.zip
vi ~/.bashrc
export JSTORM_HOME=/home/wu_being/jstorm-2.1.1
export PATH=$PATH:$JSTORM_HOME/bin

wu_being@JStorm-VirtualBox:~$ source .bashrc

3.3.1 配置$JSTORM_HOME/conf/storm.yaml

1. storm.zookeeper.servers: 表示zookeeper 的地址，
2. nimbus.host: 表示nimbus的地址
3. storm.zookeeper.root: 表示JStorm在zookeeper中的根目录，当多个JStorm共享一个zookeeper时，需要设置该选项，默认即为“/jstorm”
4. storm.local.dir: 表示JStorm临时数据存放目录，需要保证JStorm程序对该目录有写权限
5. java.library.path: Zeromq 和java zeromq library的安装目录，默认"/usr/local/lib:/opt/local/lib:/usr/lib"
6. supervisor.slots.ports: 表示Supervisor 提供的端口Slot列表，注意不要和其他端口发生冲突，默认是68xx，而Storm的是67xx
7. topology.enable.classloader: false, 默认关闭classloader，如果应用的jar与JStorm的依赖的jar发生冲突，比如应用使用thrift9，但jstorm使用thrift7时，就需要打开classloader。建议在集群级别上默认关闭，在具体需要隔离的topology上打开这个选项。

3.3.2 例子

########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
- "172.17.21.33"
storm.zookeeper.root: "/jstorm"
nimbus.host: "172.17.21.33"
nimbus.host.start.supervisor: false
nimbus.childopts: "-Xmx256m"
supervisor.childopts: "-Xmx256m"
worker.childopts: "-Xmx128m"
storm.local.dir: "%JSTORM_HOME%/data"
supervisor.slots.ports:
- 6800
- 6801
- 6802
- 6803

3.3.3 在提交jar的节点上执行

#mkdir ~/.jstorm
#cp -f $JSTORM_HOME/conf/storm.yaml ~/.jstorm

3.4 在tomcat服务器安装JStorm Web UI4

mkdir ~/.jstorm
cp -f $JSTORM_HOME/conf/storm.yaml ~/.jstorm
tar -xzf apache-tomcat-7.0.70.tar.gz   ###
cd apache-tomcat-7.0.70
cd webapps
cp $JSTORM_HOME/jstorm-ui-2.1.1.war ./
mv ROOT ROOT.old  ###
ln -s jstorm-ui-2.1.1 ROOT   ###
cd ../bin
./startup.sh  #####

.jstorm/storm.yaml

3.5 启动JStorm

nohup jstorm nimbus &

nohup jstorm supervisor &

四 更多链接

4.1 Zookeeper知识更多链接

4.2 JStorm知识更多链接