Hadoop HA环境部署

文章目录

• 1. 组件版本
• 2. 集群规划
• 3. 创建hadoop用户的目录结构(三台都执行)
• 4. 环境准备
• 4.1 修改hosts(三台都执行)
• 4.2 添加SSH信任

• 9.1 启动HDFS
• 9.2 启动Yarn
• 9.3 启动JobHistoryServer

1. 组件版本

2. 集群规划

3. 创建hadoop用户的目录结构(三台都执行)

# 添加hadoop用户
[root@spark001 ~]# useradd hadoop

# 切换至hadoop用户，创建目录
[root@spark001 ~]# su - hadoop
[hadoop@spark001 ~]$ mkdir app data lib maven_repos software script source tmp
drwxrwxr-x 2 hadoop hadoop 4096 Aug 20 22:16 app
drwxrwxr-x 2 hadoop hadoop 4096 Aug 20 22:16 data
drwxrwxr-x 2 hadoop hadoop 4096 Aug 20 22:16 lib
drwxrwxr-x 2 hadoop hadoop 4096 Aug 20 22:16 maven_repos
drwxrwxr-x 2 hadoop hadoop 4096 Aug 20 22:16 script
drwxrwxr-x 2 hadoop hadoop 4096 Aug 20 23:16 software
drwxrwxr-x 2 hadoop hadoop 4096 Aug 20 22:16 source
drwxrwxr-x 2 hadoop hadoop 4096 Aug 20 22:31 tmp

目录说明

4. 环境准备

4.1 修改hosts(三台都执行)

[root@spark001 software]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

# 将机器的内网ip添加进去
[root@spark001 software]# vim /etc/hosts
172.31.120.113 spark001
172.31.120.112 spark002
172.31.120.111 spark003

4.2 添加SSH信任

# 进入hadoop用户
[root@spark001 software]# su - hadoop

# 输入ssh-keygen，生成密钥，之后点击回车至结束
[hadoop@spark001 ~]$ ssh-keygen

# 进入.ssh文件夹，可看到公钥和私钥
[hadoop@spark001 ~]$ cd .ssh
[hadoop@spark001 .ssh]$ ll
total 8
-rw------- 1 hadoop hadoop 1679 Aug 20 22:35 id_rsa
-rw-r--r-- 1 hadoop hadoop  397 Aug 20 22:35 id_rsa.pub

# 在spark001上执行
[hadoop@spark001 .ssh]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

# 将spark002和spark003的id_rsa.pub 下载到桌面命名为id_rsa.pub2 和 id_rsa.pub3
# 将这两个文件上传到spark001的.ssh目录
[hadoop@spark001 .ssh]$ rz
[hadoop@spark001 .ssh]$ ll
total 20
-rw-rw-r-- 1 hadoop hadoop  397 Aug 20 22:40 authorized_keys
-rw------- 1 hadoop hadoop 1679 Aug 20 22:35 id_rsa
-rw-r--r-- 1 hadoop hadoop  397 Aug 20 22:35 id_rsa.pub
-rw-r--r-- 1 hadoop hadoop  397 Aug 20 22:38 id_rsa.pub2
-rw-r--r-- 1 hadoop hadoop  397 Aug 20 22:39 id_rsa.pub3

# 将这两个文件追加到authorized_keys中
[hadoop@spark001 .ssh]$ cat id_rsa.pub2 >> authorized_keys
[hadoop@spark001 .ssh]$ cat id_rsa.pub3 >> authorized_keys

#将spark001的authorized_keys下载到本地，并上传到spark002和spark003
[hadoop@spark001 .ssh]$ sz authorized_keys

# 如果有以上提示 要求输入密码 就说明有问题
[hadoop@spark001 .ssh]$ ssh spark001 date
The authenticity of host 'spark001 (172.31.120.113)' can't be established.
ECDSA key fingerprint is cf:37:5a:89:3d:68:70:56:63:e3:72:a3:c4:60:14:1a.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'spark001,172.31.120.113' (ECDSA) to the list of known hosts.
hadoop@spark001's password:

# 需要执行以下命令 给文件赋权
[hadoop@spark001 .ssh]$ chmod 600 authorized_keys

# 先在每台机器上执行ssh spark001 date，输入yes后，再执行ssh spark002 date，输入yes后，在执行ssh spark003 date
[hadoop@spark001 .ssh]$ ssh spark001 date
[hadoop@spark001 .ssh]$ ssh spark002 date
[hadoop@spark001 .ssh]$ ssh spark003 date

# 再执行下如下命令  出现时间即表示成功
ssh spark001 date
ssh spark002 date
ssh spark003 date

5. 将软件上传到spark001，再分发到spark002、spark003

# 将spark001中的软件包传到spark002、spark003
[hadoop@spark001 ~]$ cd software/
[hadoop@spark001 software]$ ll
total 438648
drwxrwxr-x 2 hadoop hadoop      4096 Aug 20 22:21 hadoop
-rw-r--r-- 1 hadoop hadoop 253270787 Aug 20 22:20 hadoop-2.6.0-cdh5.15.1.tar.gz
-rw-r--r-- 1 hadoop hadoop 191753373 Aug 20 22:27 jdk-8u191-linux-x64.tar.gz
-rw-r--r-- 1 hadoop hadoop   4142754 Aug 20 22:08 zookeeper-3.4.6.tar.gz

# 远程复制到spark002和spark003上
[hadoop@spark001 software]$ scp *.tar* spark002:`pwd`
[hadoop@spark001 software]$ scp *.tar* spark003:`pwd`

6. 安装JDK

# 退回到root用户
# 创建/usr/java目录
[root@spark001 java]# tar -zxvf /home/hadoop/software/jdk-8u191-linux-x64.tar.gz -C /usr/java/

# 权限修正
[root@spark001 java]# chown -R root:root jdk1.8.0_191
[root@spark001 java]# vim /etc/profile

#添加JAVA_HOME
export JAVA_HOME=/usr/java/jdk1.8.0_191
export PATH=$JAVA_HOME/bin:$PATH

[root@spark001 java]# source /etc/profile
[root@spark001 java]# java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)

7.安装ZooKeeper

• 解压

  # 解压
  [hadoop@spark001 software]$ tar -zxvf zookeeper-3.4.6.tar.gz -C ~/app/
  
  # 创建软链接，便于更换版本
  [hadoop@spark001 app]$ ln -s zookeeper-3.4.6 zookeeper

• 修改配置文件

  # 拷贝一份zoo_sample.cfg，命名为zoo.cfg
  [hadoop@spark001 conf]$ cp zoo_sample.cfg zoo.cfg
  [hadoop@spark001 conf]$ vim zoo.cfg
  # the directory where the snapshot is stored.
  # do not use /tmp for storage, /tmp here is just
  # example sakes.
  dataDir=/home/hadoop/data/zookeeper
  
  server.1=spark001:2888:3888
  server.2=spark002:2888:3888
  server.3=spark003:2888:3888
  
  # 创建对应目录
  [hadoop@spark001 conf]$ mkdir /home/hadoop/data/zookeeper

  以上步骤同时在三台机器执行

  # 给每台机器上的zookeeper创建唯一标识myid
  [hadoop@spark001 app]$ echo 1 > /home/hadoop/data/zookeeper/myid
  [hadoop@spark002 app]$ echo 2 > /home/hadoop/data/zookeeper/myid
  [hadoop@spark003 app]$ echo 3 > /home/hadoop/data/zookeeper/myid
  
  注意：切记给每台机器上的zookeeper创建唯一标识的时候 > 左右都需要空格

• 配置环境变量(三台同时操作)

  [hadoop@spark001 app]$ vim ~/.bash_profile
  #zookeeper
  export ZOOKEEPER_HOME=/home/hadoop/app/zookeeper
  export PATH=$ZOOKEEPER_HOME/bin:$PATH
  
  # 配置生效
  [hadoop@spark001 app]$ source ~/.bash_profile
  
  # 验证
  [hadoop@spark001 ~]$ which zkServer.sh
  ~/app/zookeeper/bin/zkServer.sh

• 启动与停止

  # 启动
  [hadoop@spark001 ~]$ zkServer.sh start
  JMX enabled by default
  Using config: /home/hadoop/app/zookeeper/bin/../conf/zoo.cfg
  Starting zookeeper ... STARTED
  
  # 查看状态
  [hadoop@spark001 ~]$ zkServer.sh status
  
  # 停止
  [hadoop@spark001 ~]$ zkServer.sh stop

8. 安装Hadoop

• 解压

  # 解压至app目录
  [hadoop@spark001 software]$ tar -zxvf hadoop-2.6.0-cdh5.15.1.tar.gz -C ~/app/
  
  # 软链接，便于更换版本
  [hadoop@spark001 app]$ ln -s hadoop-2.6.0-cdh5.15.1 hadoop

• 修改配置文件

  # 修改hadoop-env.sh，添加JAVA_HOME
  # The java implementation to use.
  export JAVA_HOME=/usr/java/jdk1.8.0_191
  
  # 删除 core-site.xml、hdfs-site.xml、yarn-site.xml，上传已修改好的对应配置
  [hadoop@spark001 hadoop]$ rm -f core-site.xml hdfs-site.xml yarn-site.xml
  
  ## 上传好之后 ll 查看下文件的角色信息 如果不是hadoop用户 修改下
  [hadoop@spark001 hadoop]$ chown -R hadoop:hadoop /home/hadoop/app/hadoop/etc/hadoop

• 配置环境变量

  [hadoop@spark001 hadoop]$ vim ~/.bash_profile
  #hadoop
  export HADOOP_HOME=/home/hadoop/app/hadoop
  export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
  [hadoop@spark001 hadoop]$ source ~/.bash_profile

9. 启动集群

9.1 启动HDFS

• 启动之前需要先启动JournalNode

  # 启动JournalNode(三台都执行)
  [hadoop@spark001 sbin]$ hadoop-daemon.sh start journalnode
  
  # jps查看下启动情况
  [hadoop@spark001 sbin]$ jps
  2181 QuorumPeerMain
  2302 JournalNode
  2351 Jps

• 格式化NameNode

  # 在spark001上执行
  [hadoop@spark001 data]$ hadoop namenode -format
  
  # 看到这句话说明初始化成功
  Storage directory /home/hadoop/data/dfs/name has been successfully formatted.
  
  # 将初始化的namenode信息同步到第二台机器上 为了让两台namenode信息保持一致
  [hadoop@spark001 dfs]$ scp -r name spark002:`pwd`

• 初始化ZKFC

  # 初始化ZKFC只需要选择一台机器即可，因为zk是集群，选择一台建立hdfs zk之间的关联关系就行了。出现以下信息及表示初始化成功。
  [hadoop@spark001 sbin]$ hdfs zkfc -formatZK
  19/08/23 15:30:37 INFO ha.ActiveStandbyElector: Session connected.
  19/08/23 15:30:37 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/ruozeclusterg7 in ZK.
  19/08/23 15:30:37 INFO zookeeper.ZooKeeper: Session: 0x26cbd19119a0000 closed
  19/08/23 15:30:37 INFO zookeeper.ClientCnxn: EventThread shut down
  19/08/23 15:30:37 INFO tools.DFSZKFailoverController: SHUTDOWN_MSG:

• 启动HDFS

  [hadoop@spark001 ~]$ start-dfs.sh
  Starting namenodes on [spark001 spark002]
  spark001: starting namenode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/hadoop-hadoop-namenode-spark001.out
  spark002: starting namenode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/hadoop-hadoop-namenode-spark002.out
  spark001: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/hadoop-hadoop-datanode-spark001.out
  spark002: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/hadoop-hadoop-datanode-spark002.out
  spark003: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/hadoop-hadoop-datanode-spark003.out
  Starting journal nodes [spark001 spark002 spark003]
  spark002: starting journalnode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/hadoop-hadoop-journalnode-spark002.out
  spark003: starting journalnode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/hadoop-hadoop-journalnode-spark003.out
  spark001: starting journalnode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/hadoop-hadoop-journalnode-spark001.out
  Starting ZK Failover Controllers on NN hosts [spark001 spark002]
  spark001: starting zkfc, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/hadoop-hadoop-zkfc-spark001.out
  spark002: starting zkfc, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/hadoop-hadoop-zkfc-spark002.out
  
  # 验证一下
  [hadoop@spark001 ~]$ jps
  2352 NameNode
  2838 DFSZKFailoverController
  2457 DataNode
  2651 JournalNode
  2940 Jps
  2190 QuorumPeerMain

• 单个进程启动

  # namenode datanode journalnode zkfc
  [hadoop@spark001 ~]$ hadoop-daemon.sh start namenode

• 防止被挖矿设置

  
  
  

9.2 启动Yarn

• 启动

  # 在spark001上执行
  [hadoop@spark001 ~]$ start-yarn.sh
  starting yarn daemons
  starting resourcemanager, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/yarn-hadoop-resourcemanager-spark001.out
  spark003: starting nodemanager, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/yarn-hadoop-nodemanager-spark003.out
  spark002: starting nodemanager, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/yarn-hadoop-nodemanager-spark002.out
  spark001: starting nodemanager, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.15.1/logs/yarn-hadoop-nodemanager-spark001.out
  
  # spark002上的resourcemanager需要单独手动开启
  [hadoop@spark002 ~]$ yarn-daemon.sh start resourcemanager
  
  # 此时三台机器下的进程
  # spark001
  [hadoop@spark001 ~]$ jps
  2352 NameNode
  3089 ResourceManager
  3523 Jps
  3187 NodeManager
  2838 DFSZKFailoverController
  2457 DataNode
  2651 JournalNode
  2190 QuorumPeerMain
  
  # spark002
  [hadoop@spark002 ~]$ jps
  2544 DFSZKFailoverController
  3139 ResourceManager
  2324 DataNode
  2420 JournalNode
  2165 QuorumPeerMain
  3206 Jps
  2251 NameNode
  2958 NodeManager
  
  # spark003
  [hadoop@spark003 ~]$ jps
  2353 JournalNode
  2257 DataNode
  2165 QuorumPeerMain
  2619 Jps
  2493 NodeManager

• web页面访问

  # 1.对于HDFS来说，访问web页面
  NameNode(active)：spark001(公网):50070
  NameNode(standby)：spark001(公网):50070
  
  #2.对于Yarn
  ResourceManager(active)：spark001(公网):8088
  ResourceManager(standby)：spark001(公网):8088/cluster/cluster

9.3 启动JobHistoryServer

作用：作业的历史查看

• 启动

  # 在spark001上执行
  [hadoop@spark001 sbin]$ mr-jobhistory-daemon.sh start historyserver
  [hadoop@spark001 sbin]$ jps
  2352 NameNode
  3089 ResourceManager
  3187 NodeManager
  2838 DFSZKFailoverController
  3704 Jps
  2457 DataNode
  2651 JournalNode
  2190 QuorumPeerMain
  3631 JobHistoryServer   # 启动成功

• web页面访问：spark001(公网ip):19888
  

10. 关闭集群

关闭顺序是跟启动顺序反着的

# 在saprk001上执行
[hadoop@spark001 sbin]$ mr-jobhistory-daemon.sh stop historyserver

# 在spark002上执行
[hadoop@spark002 sbin]$ yarn-daemon.sh stop resourcemanager

# 在saprk001上执行
[hadoop@spark001 sbin]$ stop-all.sh

# 在三台机器同时执行
[hadoop@spark001 sbin]$ zkServer.sh stop
[hadoop@spark002 sbin]$ zkServer.sh stop
[hadoop@spark003 sbin]$ zkServer.sh stop

参考博客

• 点赞
• 收藏
• 分享
• 文章举报