Linux Centos7 环境搭建Docker部署Zookeeper分布式集群服务实战
Zookeeper完全分布式集群服务
准备好3台服务器:
- [x]A-> centos-helios:192.168.19.1
- [x]B-> centos-hestia:192.168.19.2
[x]C-> centos-hebe:192.168.19.3
⚠️ [注意事项]:
1.一定要确定3台服务器之间可以相互通信
2.如果是采用的阿里云等服务器的话,一定要配置主机名与IP映射,否则会连接拒绝等问题
配置Zookeeper安装目录
在3台宿主机配置zookeeper安装目录:/docker/zookeeper
- 并且在文件夹创建 data 和logs 目录:mkdir -p /docker/zookeeper/data && mkdir -p /docker/zookeeper/logs
mkdir -p /docker/zookeeper/data mkdir -p /docker/zookeeper/logs
- 授权文件目录:chmod -R 777 /docker/zookeeper/
安装和部署Zookeeper
拉取zookeeper镜像:docker pull zookeeper:3.5.6
[root@centos-helios ~]# docker pull zookeeper:3.5.6 3.5.6: Pulling from library/zookeeper 8ec398bc0356: Pull complete 8e866b095956: Pull complete e18af1784b1f: Pull complete 9cec9f51936c: Pull complete 475dcbe0e2e9: Pull complete 222ebb7346b3: Pull complete f0552f5b3f6c: Pull complete cdfcdbc43b13: Pull complete Digest: sha256:6b6b5f7fb6a47d2b311df5af1718af5a425a679dbb844d77913fa68d1a8bf0fd Status: Downloaded newer image for zookeeper:3.5.6 docker.io/library/zookeeper:3.5.6 [root@centos-helios ~]#
⚠️ [注意事项]:
1.由于Zookeeper是一个常规的基础服务组件,我这直接拉取Docker hub镜像,有兴趣的可以自行构建镜像部署
2.这里只对centos-helios有举例,另外2台需要执行相同操作,以下不再赘述
临时安装Zookeeper服务:zookeeper-tmp-server
docker run -itd -p 2181:2181 --restart always --name=zookeeper-tmp-server --privileged=true \ -v /docker/develop/zookeeper/data:/data \ -v /docker/develop/zookeeper/datalog:/datalog \ -v /docker/develop/zookeeper/logs:/logs \ -e "ZOO_STANDALONE_ENABLED=true" \ -e "TZ=Asia/Shanghai" \ -e "ZOO_4LW_COMMANDS_WHITELIST=*" \ -e "ZOO_AUTOPURGE_SNAPRETAINCOUNT=5" \ -e "ZOO_AUTOPURGE_PURGEINTERVAL=24" \ -e "ZOO_MAX_CLIENT_CNXNS=64" \ -e "ZOO_TICK_TIME=2000" \ -e "ZOO_INIT_LIMIT=100" \ -e "ZOO_SYNC_LIMIT=5" \ zookeeper:3.5.6
复制zookeeper-tmp-server的conf到/docker/zookeeper:
1.查看容器列表:docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Ports}}"
[root@centos-helios zookeeper]# docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Ports}}" CONTAINER ID NAMES PORTS 71ad055739b6 zookeeper-tmp-server 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, 8080/tcp [root@centos-helios zookeeper]#
2.执行Docker cp 命令:docker cp Zookeeper容器ID:/conf /docker/zookeeper/
[root@centos-helios zookeeper]# docker cp 71ad055739b6:/conf /docker/zookeeper/ [root@centos-helios zookeeper]#
centos-helios:docker cp 71ad055739b6:/conf /docker/zookeeper/
centos-hebe:docker cp bf9df9966efb:/conf /docker/zookeeper/
centos-hestia:docker cp 4b4f8e813c80:/conf /docker/zookeeper/
编写zookeeper的zoo.cfg配置文件:
1.服务器集群配置:
- [x]centos-helios:192.168.19.1:2888:3888
- [x]centos-hestia:192.168.19.2:2888:3888
- [x]centos-hebe:192.168.19.3:2888:3888
2.对应的配置文件配置如下:
- [x]centos-helios:/docker/zookeeper/conf/zoo.cfg
tickTime=2000 initLimit=100 syncLimit=5 dataDir=/data dataLogDir=/datalog clientPort=2181 quorumListenOnAllIPs=true preAllocSize=64M globalOutstandingLimit=100000 snapCount=50000 maxClientCnxns=64 minSessionTimeout=4000 maxSessionTimeout=40000 4lw.commands.whitelist=* admin.enableServer=true server.1=192.168.19.1:2888:3888 server.2=192.168.19.2:2888:3888 server.3=192.168.19.3:2888:3888 tcpKeepAlive=true ipReachableTimeout=0 cnxTimeout=3 electionAlg=3 autopurge.snapRetainCount=5 autopurge.purgeInterval=24 #忽略ACL验证 skipACL=yes forceSync=yes
- [x]centos-hestia:/docker/zookeeper/conf/zoo.cfg
tickTime=2000 initLimit=100 syncLimit=5 dataDir=/data dataLogDir=/datalog clientPort=2181 quorumListenOnAllIPs=true preAllocSize=64M globalOutstandingLimit=100000 snapCount=50000 maxClientCnxns=64 minSessionTimeout=4000 maxSessionTimeout=40000 4lw.commands.whitelist=* admin.enableServer=true server.1=192.168.19.1:2888:3888 server.2=192.168.19.2:2888:3888 server.3=192.168.19.3:2888:3888 tcpKeepAlive=true ipReachableTimeout=0 cnxTimeout=3 electionAlg=3 autopurge.snapRetainCount=5 autopurge.purgeInterval=24 #忽略ACL验证 skipACL=yes forceSync=yes
- [x]centos-hebe:/docker/zookeeper/conf/zoo.cfg
tickTime=2000 initLimit=100 syncLimit=5 dataDir=/data dataLogDir=/datalog clientPort=2181 quorumListenOnAllIPs=true preAllocSize=64M globalOutstandingLimit=100000 snapCount=50000 maxClientCnxns=64 minSessionTimeout=4000 maxSessionTimeout=40000 4lw.commands.whitelist=* admin.enableServer=true server.1=192.168.19.1:2888:3888 server.2=192.168.19.2:2888:3888 server.3=192.168.19.3:2888:3888 tcpKeepAlive=true ipReachableTimeout=0 cnxTimeout=3 electionAlg=3 autopurge.snapRetainCount=5 autopurge.purgeInterval=24 #忽略ACL验证 skipACL=yes forceSync=yes
⚠️ [注意事项]:如果是采用阿里云等云服务器部署,一般设置为:
server.1=阿里云[外网IP OR 内网IP]:2888:3888 server.2=阿里云[外网IP OR 内网IP]:2888:3888 server.3=阿里云[外网IP OR 内网IP]:2888:3888
编写正式安装Dcoekr命令:
- [x]centos-helios:
docker run -itd -p 2181:2181 -p 2888:2888 -p 3888:3888 --restart always --name=zookeeper-server --hostname zookeeper-server --privileged=true \ -v /docker/zookeeper/conf/zoo.cfg:/conf/zoo.cfg \ -v /docker/zookeeper/data:/data \ -v /docker/zookeeper/datalog:/datalog \ -v /docker/zookeeper/logs:/logs \ -e "TZ=Asia/Shanghai" \ -e "ZOO_MY_ID=1" \ -e "JAVA_OPTS=-server -Xms512m -Xmx512m -Xmn256m -Duser.home=/opt -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:+AlwaysPreTouch -XX:-UseBiasedLocking" \ zookeeper:3.5.6
- [x]centos-hestia:
docker run -itd -p 2181:2181 -p 2888:2888 -p 3888:3888 --restart always --name=zookeeper-server --hostname zookeeper-server --privileged=true \ -v /docker/zookeeper/conf/zoo.cfg:/conf/zoo.cfg \ -v /docker/zookeeper/data:/data \ -v /docker/zookeeper/datalog:/datalog \ -v /docker/zookeeper/logs:/logs \ -e "TZ=Asia/Shanghai" \ -e "ZOO_MY_ID=2" \ -e "JAVA_OPTS=-server -Xms512m -Xmx512m -Xmn256m -Duser.home=/opt -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:+AlwaysPreTouch -XX:-UseBiasedLocking" \ zookeeper:3.5.6
- [x]centos-hebe:
docker run -itd -p 2181:2181 -p 2888:2888 -p 3888:3888 --restart always --name=zookeeper-server --hostname zookeeper-server --privileged=true \ -v /docker/zookeeper/conf/zoo.cfg:/conf/zoo.cfg \ -v /docker/zookeeper/data:/data \ -v /docker/zookeeper/datalog:/datalog \ -v /docker/zookeeper/logs:/logs \ -e "TZ=Asia/Shanghai" \ -e "ZOO_MY_ID=3" \ -e "JAVA_OPTS=-server -Xms512m -Xmx512m -Xmn256m -Duser.home=/opt -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:+AlwaysPreTouch -XX:-UseBiasedLocking" \ zookeeper:3.5.6
后续服务配置
1.开放端口[2181 2888 3888]
- [1].启动防火墙:systemctl start firewalld.service
- [2].查看防火墙状态:systemctl status firewalld.service
- [3].开放端口:firewall-cmd --zone=public --add-port=端口号/tcp --permanent
firewall-cmd --zone=public --add-port=2181/tcp --permanent firewall-cmd --zone=public --add-port=2888/tcp --permanent firewall-cmd --zone=public --add-port=3888/tcp --permanent
- [4].加载初始化防火墙生效:firewall-cmd --reload
- [5].停用防火墙:systemctl stop firewalld.service
⚠️ [注意事项]:
1.如果是Centos7以及以上系统,防火墙默认是firewall,对于Centos7版本请自行进行配置
2.依据实测,对于2888和3888等端口必须开放,否则在使用Zookeeper服务的时候会报连接[KeeperErrorCode = ConnectionLoss]错误
2.配置启动以及测试zookeeper服务:
- [x]启动容器Zookeeper服务顺序:A->B->C-重启A
- [x]进入容器输入:bin/zkServer.sh status
centos-helios:
root@zookeeper-server:/apache-zookeeper-3.5.6-bin# bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /conf/zoo.cfg Client port found: 2181. Client address: localhost. Mode: follower root@zookeeper-server:/apache-zookeeper-3.5.6-bin#
centos-hestia:
root@zookeeper-server:/apache-zookeeper-3.5.6-bin# bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /conf/zoo.cfg Client port found: 2181. Client address: localhost. Mode: leader root@zookeeper-server:/apache-zookeeper-3.5.6-bin#
centos-hebe:
root@zookeeper-server:/apache-zookeeper-3.5.6-bin# bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /conf/zoo.cfg Client port found: 2181. Client address: localhost. Mode: follower root@zookeeper-server:/apache-zookeeper-3.5.6-bin#
⚠️ [注意事项]:
1.Zookeeper的集群服务采用的是Leader-Follower线程模型,如上所述。
2.Zookeeper使用了ZAB(Zookeeper Atomic Broadcast)协议,保证了leader,follower的一致性,leader 负责数据的读写,而follower只负责数据的读,如果follower遇到写操作,会提交到leader。
版权声明:本文为博主原创文章,遵循相关版权协议,如若转载或者分享请附上原文出处链接和链接来源。
- 点赞
- 收藏
- 分享
- 文章举报
- docker简易环境搭建实战篇:以人人网开源项目(renren-fast)搭建分布式部署(后端篇)
- 生产环境实战spark (11)分布式集群 5台设备 Zookeeper集群、Kafka集群安装部署
- 【Docker】「实战篇」开源项目docker化运维部署-搭建mysql集群(四)
- zookeeper伪分布式集群环境搭建
- zookeeper部署及集群环境搭建
- 大数据分布式集群组件--Zookeeper详解及环境搭建
- 分布式集群环境hadoop1.2.0、hbase0.94.4、zookeeper、elasticsearch搭建2
- 基于Mesos+Marathon+Zookeeper的Docker集群管理环境搭建过程分享
- Hadoop2.7.3+HBase1.2.5+ZooKeeper3.4.6搭建分布式集群环境
- Hadoop2.x部署之完全分布式集群环境搭建
- 自己学Docker:15.Docker使用实战--zookeeper集群部署
- hadoop-2.7.4+hbase-1.3.1+zookeeper-3.4.9搭建分布式集群环境
- 分布式集群环境hadoop、hbase、zookeeper搭建(全)
- ZooKeeper实战(一)Zookeeper伪集群环境搭建
- ZooKeeper一二事 - 搭建ZooKeeper伪分布式及正式集群 提供集群服务
- 基于dubbo从传统MVC架构转向SOA架构分布式设计4--(服务部署集群搭建及负载均衡)
- 阿里云ECS服务器部署HADOOP集群(一):Hadoop完全分布式集群环境搭建
- hadoop-2.7.4+hbase-1.3.1+zookeeper-3.4.9搭建分布式集群环境
- mesos+marathon+zookeeper的docker管理集群亲手搭建实例(环境Centos6.8)
- 配置密码分布式集群环境hadoop、hbase、zookeeper搭建(全)