分布式消息队列RocketMQ部署与监控

摘要:分布式消息队列RocketMQ部署与监控

==========================================================================================

一、RocketMQ简介

==========================================================================================

RocketMQ是一款分布式、队列模型的消息中间件，具有以下特点：

1、支持严格的消息顺序；

2、支持Topic与Queue两种模式；

3、亿级消息堆积能力；

4、比较友好的分布式特性；

5、同时支持Push与Pull方式消费消息；

==========================================================================================

二、服务器分布

==========================================================================================

1、相关说明

IP地址	主机名	机型	角色	架构模式
10.217.121.123	rocketmq-master1	128G内存	nameserver、brokerserver	Master1（双Master模式）
10.217.122.31	rocketmq-master2	128G内存	nameserver、brokerserver	Master2（双Master模式）

2、hosts信息添加

#vim/etc/hosts

10.217.121.123mqnameserver1

10.217.122.31mqnameserver2

10.217.121.123rocketmq-master1

10.217.122.31rocketmq-master2

3、系统环境

CentOS6.3

4、总体架构





==========================================================================================

三、RocketMQ安装与配置

==========================================================================================

1、JDK安装

http://download.oracle.com/otn-pub/java/jdk/7u65-b17/jdk-7u65-linux-x64.tar.gz

#tarxvzfjdk-7u65-linux-x64.gz-C/usr/local

#rm-fjdk-7u65-linux-x64.gz

#cd/usr/local&&ln-sjdk1.7.0_65jdk

#cd/usr/local/bin&&ln-s/usr/local/jdk/bin/java

2、RocketMQ安装

https://github.com/alibaba/RocketMQ/releases

#tarxvzfalibaba-rocketmq-3.1.7.tar.gz-C/usr/local

#mvalibaba-rocketmqalibaba-rocketmq-3.1.7

#ln-salibaba-rocketmq-3.1.7rocketmq

#rmalibaba-rocketmq-3.1.7.tar.gz

#ll/usr/local





3、环境变量设置

#vim/etc/profile

exportPATH=$PATH:/usr/local/bin

exportJAVA_HOME=/usr/local/jdk

exportCLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

exportPATH=$PATH:$JAVA_HOME/bin

exportROCKETMQ_HOME=/usr/local/rocketmq

exportPATH=$PATH::$ROCKETMQ_HOME/bin

#source/etc/profile

#cd/usr/local/rocketmq/bin&&shos.sh

4、主机名设置

(1)、Master1服务器

#vim/etc/hosts

10.217.121.123mqnameserver1

10.217.122.31mqnameserver2

10.217.121.123rocketmq-master1

10.217.122.31rocketmq-master2

#sed-i'/HOSTNAME/d'/etc/sysconfig/network

#echo'HOSTNAME=rocketmq-master1'>>/etc/sysconfig/network

#hostnamerocketmq-master1

(2)、Master2服务器

#vim/etc/hosts

10.217.121.123mqnameserver1

10.217.122.31mqnameserver2

10.217.121.123rocketmq-master1

10.217.122.31rocketmq-master2

#sed-i'/HOSTNAME/d'/etc/sysconfig/network

#echo'HOSTNAME=rocketmq-master2'>>/etc/sysconfig/network

#hostnamerocketmq-master2

5、RocketMQ配置

(1)、Master1服务器

#vim/usr/local/rocketmq/conf/2m-noslave/broker-a.properties

brokerClusterName=AdpMqCluster

brokerName=broker-a

brokerId=0

namesrvAddr=mqnameserver1:9876;mqnameserver2:9876

defaultTopicQueueNums=4

autoCreateTopicEnable=true

autoCreateSubscriptionGroup=true

listenPort=10911

deleteWhen=04

fileReservedTime=120

mapedFileSizeCommitLog=1073741824

mapedFileSizeConsumeQueue=50000000

destroyMapedFileIntervalForcibly=120000

redeleteHangedFileInterval=120000

diskMaxUsedSpaceRatio=88

storePathRootDir=/data/rocketmq/store

storePathCommitLog=/data/rocketmq/store/commitlog

maxMessageSize=65536

flushCommitLogLeastPages=4

flushConsumeQueueLeastPages=2

flushCommitLogThoroughInterval=10000

flushConsumeQueueThoroughInterval=60000

brokerRole=ASYNC_MASTER

flushDiskType=ASYNC_FLUSH

checkTransactionMessageEnable=false

sendMessageThreadPoolNums=128

pullMessageThreadPoolNums=128

(2)、Master2服务器

#vim/usr/local/rocketmq/conf/2m-noslave/broker-b.properties

brokerClusterName=AdpMqCluster

brokerName=broker-b

brokerId=1

namesrvAddr=mqnameserver1:9876;mqnameserver2:9876

defaultTopicQueueNums=4

autoCreateTopicEnable=true

autoCreateSubscriptionGroup=true

listenPort=10911

deleteWhen=04

fileReservedTime=120

mapedFileSizeCommitLog=1073741824

mapedFileSizeConsumeQueue=50000000

destroyMapedFileIntervalForcibly=120000

redeleteHangedFileInterval=120000

diskMaxUsedSpaceRatio=88

storePathRootDir=/data/rocketmq/store

storePathCommitLog=/data/rocketmq/store/commitlog

maxMessageSize=65536

flushCommitLogLeastPages=4

flushConsumeQueueLeastPages=2

flushCommitLogThoroughInterval=10000

flushConsumeQueueThoroughInterval=60000

brokerRole=ASYNC_MASTER

flushDiskType=ASYNC_FLUSH

checkTransactionMessageEnable=false

sendMessageThreadPoolNums=128

pullMessageThreadPoolNums=128

(3)、runbroker.sh参数调整

runbroker.sh需要根据内存大小进行适当地调整

JAVA_OPT_1="-server-Xms80g-Xmx80g-Xmn20g-XX:PermSize=1g-XX:MaxPermSize=1g"

6、服务启动

#mkdir-p/data/rocketmq/store/commitlog/data/logs

#cd/usr/local/rocketmq/conf&&sed-i's#${user.home}#/data#g'*.xml

(1)、启动NameServer【master1、master2】

#cd/usr/local/rocketmq/bin

#nohupshmqnamesrv&

#nohupshmqnamesrv>/var/log/ns.log& #nohupshmqbroker-c../conf/2m-noslave/broker-a.properties>/var/log/mq.log2>&1&

(2)、启动BrokerServerA【master1】

#cd/usr/local/rocketmq/bin

#nohupshmqbroker-c../conf/2m-noslave/broker-a.properties>/dev/null2>&1&

(3)、启动BrokerServerB【master2】

#cd/usr/local/rocketmq/bin

#nohupshmqbroker-c../conf/2m-noslave/broker-b.properties>/dev/null2>&1&

#netstat-ntlp





#jps





==========================================================================================

四、RocketMQ服务健康监控

==========================================================================================

1、依赖组件安装

#yum-yinstallnmap

2、nameserver监控

#vim/data/scripts/check_nameserver_health.sh

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

#!/bin/sh PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin SRV_PORT="9876"##端口号 SRV_PROT="tcp"##协议类型 SRV_NAME="rocketmq_nameserver"##服务名 ##是否已正确扫描 SCAN_FLAG=0 forETH_NAMEin`/sbin/ifconfig|grepeth1|awk'{print$1}'` do ETH1_ADDR=`/sbin/ifconfig${ETH_NAME}|awk-F':''/inetaddr/{print$2}'|sed's/[a-zA-Z]//g'` [[-z"${ETH1_ADDR}"]]&&continue||break done TMP_SRV_PROT=`echo${SRV_PROT}|tr'[A-Z]''[a-z]'` if[["${TMP_SRV_PROT}"=="tcp"]];then PROT_OPT="S" elif[["${TMP_SRV_PROT}"=="udp"]];then PROT_OPT="U" else echo"未知的协议类型！"&&exit1 fi ##最多扫描3次，成功一次即可，以避免网络抖动而导致误判 for((i=0;i<3;i++));do RETVAL=`/usr/bin/nmap-n-s${PROT_OPT}-p${SRV_PORT}${ETH1_ADDR}|grepopen` [[-n"${RETVAL}"]]&&SCAN_FLAG=1;break||sleep10 done if[[${SCAN_FLAG}-ne1]];then [[-n`psaux|grepjava|grepnamesrv`]]&&kill-9`psaux|grepjava|awk'/namesrv/{print$2}'` cd/usr/local/rocketmq/bin&&nohupshmqnamesrv& fi

3、brokerserver监控

#vim/data/scripts/check_brokerserver_health.sh

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

#!/bin/sh PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin SRV_PORT="10911"##端口号 SRV_PROT="tcp"##协议类型 SRV_NAME="rocketmq_brokerserver"##服务名 ##是否已正确扫描 SCAN_FLAG=0 forETH_NAMEin`/sbin/ifconfig|grepeth1|awk'{print$1}'` do ETH1_ADDR=`/sbin/ifconfig${ETH_NAME}|awk-F':''/inetaddr/{print$2}'|sed's/[a-zA-Z]//g'` [[-z"${ETH1_ADDR}"]]&&continue||break done TMP_SRV_PROT=`echo${SRV_PROT}|tr'[A-Z]''[a-z]'` if[["${TMP_SRV_PROT}"=="tcp"]];then PROT_OPT="S" elif[["${TMP_SRV_PROT}"=="udp"]];then PROT_OPT="U" else echo"未知的协议类型！"&&exit1 fi ##最多扫描3次，成功一次即可，以避免网络抖动而导致误判 for((i=0;i<3;i++));do RETVAL=`/usr/bin/nmap-n-s${PROT_OPT}-p${SRV_PORT}${ETH1_ADDR}|grepopen` [[-n"${RETVAL}"]]&&SCAN_FLAG=1;break||sleep10 done if[[${SCAN_FLAG}-ne1]];then [[-n`psaux|grepjava|grepbroker`]]&&kill-9`psaux|grepjava|awk'/broker/{print$2}'` cd/usr/local/rocketmq/bin&&nohupshmqbroker-c../conf/2m-noslave/broker-a.properties& fi

【注意】

以上红色标注部分，在master2上的名称为“broker-b.properties”

4、crontab信息添加

#touch/var/run/check_rocketmq_nameserver.lock

#touch/var/run/check_rocketmq_brokerserver.lock

#echo'touch/var/run/check_rocketmq_nameserver.lock'>>/etc/rc.d/rc.local

#echo'touch/var/run/check_rocketmq_brokerserver.lock'>>/etc/rc.d/rc.local

#crontab-e

*/2****(flock--timeout=0/var/run/check_rocketmq_nameserver.lock/data/scripts/check_nameserver_health.sh>/dev/null2>&1)

*/2****(flock--timeout=0/var/run/check_rocketmq_brokerserver.lock/data/scripts/check_brokerserver_health.sh>/dev/null2>&1)