您的位置:首页 > 运维架构

hadoop集群启动脚本——解决启动hadoop集群时,效率低问题

2017-06-28 19:12 465 查看

解决启动hadoop集群时,效率低问题

当搭建好了hadoop集群时,需要先启动协调zookeeper,再启动hdfs,最后启动yarn,很麻烦呀,效率太低了,有木有!!!!!!!!!!

当然,前题是搭建起来了hadoop集群,可以参考我搭建的hadoop集群

1.参考zookeeper搭建

2.参考hadoop集群搭建

以下是针对我自己搭建的hadoop集群,编写的启动脚本

#!/bin/bash
#我们将脚本存入在itcast01,因为它有到02、03、04、05、06的免登录

clear
echo "->->->->->->->->->->->->start zkServer-----------------------------------"
for ip in itcast04 itcast05 itcast06
do
echo "ssh to $ip"
ssh root@$ip "source /etc/profile;/itcast/zookeeper-3.4.6/bin/zkServer.sh start"
ssh root@$ip "source /etc/profile;jps"
done

echo "->->->->->->->->->->->->start itcast01 itcast02 hdfs---------------------"
ip=itcast01
ssh root@$ip "source /etc/profile;/itcast/hadoop-2.2.0/sbin/start-dfs.sh"
ip=itcast01
ssh root@$ip "source /etc/profile;jps"
ip=itcast02
ssh root@$ip "source /etc/profile;jps"

echo "->->->->->->->->->->->->start itcast03 yarn------------------------------"
ip=itcast03
ssh root@$ip "source /etc/profile;/itcast/hadoop-2.2.0/sbin/start-yarn.sh"
ssh root@$ip "source /etc/profile;jps"


执行过程如下:

[root@itcast01 ~]# ./start_hadoop.sh

->->->->->->->->->->->->start zkServer-----------------------------------
ssh to itcast04
JMX enabled by default
Using config: /itcast/zookeeper-3.4.5/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
4049 Jps
4016 QuorumPeerMain
ssh to itcast05
JMX enabled by default
Using config: /itcast/zookeeper-3.4.5/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
4114 QuorumPeerMain
4150 Jps
ssh to itcast06
JMX enabled by default
Using config: /itcast/zookeeper-3.4.5/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
4017 Jps
3977 QuorumPeerMain
->->->->->->->->->->->->start itcast01 itcast02 hdfs---------------------
Starting namenodes on [itcast01 itcast02]
itcast01: starting namenode, logging to /itcast/hadoop-2.2.0/logs/hadoop-root-namenode-itcast01.out
itcast02: starting namenode, logging to /itcast/hadoop-2.2.0/logs/hadoop-root-namenode-itcast02.out
itcast05: starting datanode, logging to /itcast/hadoop-2.2.0/logs/hadoop-root-datanode-itcast05.out
itcast04: starting datanode, logging to /itcast/hadoop-2.2.0/logs/hadoop-root-datanode-itcast04.out
itcast06: starting datanode, logging to /itcast/hadoop-2.2.0/logs/hadoop-root-datanode-itcast06.out
Starting journal nodes [itcast04 itcast05 itcast06]
itcast05: starting journalnode, logging to /itcast/hadoop-2.2.0/logs/hadoop-root-journalnode-itcast05.out
itcast06: starting journalnode, logging to /itcast/hadoop-2.2.0/logs/hadoop-root-journalnode-itcast06.out
itcast04: starting journalnode, logging to /itcast/hadoop-2.2.0/logs/hadoop-root-journalnode-itcast04.out
Starting ZK Failover Controllers on NN hosts [itcast01 itcast02]
itcast02: starting zkfc, logging to /itcast/hadoop-2.2.0/logs/hadoop-root-zkfc-itcast02.out
itcast01: starting zkfc, logging to /itcast/hadoop-2.2.0/logs/hadoop-root-zkfc-itcast01.out
7702 DFSZKFailoverController
7763 Jps
7431 NameNode
5695 NameNode
5794 DFSZKFailoverController
5863 Jps
->->->->->->->->->->->->start itcast03 yarn------------------------------
starting yarn daemons
starting resourcemanager, logging to /itcast/hadoop-2.2.0/logs/yarn-root-resourcemanager-itcast03.out
itcast04: starting nodemanager, logging to /itcast/hadoop-2.2.0/logs/yarn-root-nodemanager-itcast04.out
itcast06: starting nodemanager, logging to /itcast/hadoop-2.2.0/logs/yarn-root-nodemanager-itcast06.out
itcast05: starting nodemanager, logging to /itcast/hadoop-2.2.0/logs/yarn-root-nodemanager-itcast05.out
itcast03: starting nodemanager, logging to /itcast/hadoop-2.2.0/logs/yarn-root-nodemanager-itcast03.out
4575 NodeManager
4464 ResourceManager
4792 Jps
[root@itcast01 ~]#

到这里了,恭喜你,成功启动hadoop集群!!!

hadoop集群停止运行脚本链接
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息