hadoop-2.5.0-cdh5.3.0 HA在线升级

本文升级到hadoop2.6

1、hadoop升级前准备，namenode元数据和配置备份等信息参考上篇文章：http://blog.csdn.net/linux_ja/article/details/51908466

2、备份信息：

hdfs dfsadmin -rollingUpgrade prepare

hdfs dfsadmin -rollingUpgrade query

3、进入standby nn节点执行如下四个命令：

sudo rm -rf /opt/hadoop

sudo ln -s /home/hadoop/platform/hadoop-2.6.0-cdh5.4.0 /opt/hadoop

hadoop-daemon.sh stop namenode

nohup hdfs namenode -rollingUpgrade started &

注意，目前hdfs namenode -rollingUpgrade started这个命令先以后台运行，过一段时间以正常命令启动nn，

4、进入active nn节点重复执行步骤2的命令，这样就完成nn的在线升级

5、在nn节点上以此执行所有要升级的datanode，我的四个节点(hdfs.beta1,hdfs.beta2,hdfs.beta3,hdfs.beta4)

 a、hdfs dfsadmin -getDatanodeInfo hdfs.beta1:50020

 b、hdfs dfsadmin -shutdownDatanode hdfs.beta1:50020 upgrade

  c、到所要升级的datanode上执行，hadoop-daemon.sh start datanode

以此重复执行abc步骤，这就完成dn的在线升级步骤

6、如果允许一段时间没问题，可以执行：

hdfs dfsadmin -rollingUpgrade finalize，执行后就不能回滚

7、test：

hadoop jar /opt/hadoop/share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.4.0.jar randomwriter /rand 

hadoop jar /opt/hadoop/share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.4.0.jar sort rand /sort-rand 

hadoop jar /opt/hadoop/share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.4.0.jar wordcount  /wordcount/a.txt /wordcount/temp

8、升级之后问题：

org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: 

got premature end-of-file at txid 445259; expected file to go up to 4243581

nn和standby nn的editlog不一致

执行：hdfs namenode -bootstrapStandby