Hadoop Hdfs DataNode 启动失败

FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for block pool Block pool BP-510292565-192.168.246.128-1480349157516 (storage id DS-1215049814-192.168.88.130-50010-1455139414431) service to node2/192.168.246.131:8020

java.io.IOException: Incompatible clusterIDs in /opt/hadoop2/dfs/data: namenode clusterID = CID-14cfdb37-d378-4aa5-815b-2c84417ddbf9; datanode clusterID = CID-584a9b34-9a97-4155-8373-8b285674e9f3
at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:391)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:191)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:219)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:837)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:808)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:280)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:222)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:664)

at java.lang.Thread.run(Thread.java:745)

出现这个问题的原因是DataNode的clusterID和NameNode的clusterID不匹配.   --clusterID的作用是唯一标识一个集群.

解决办法是,将name/current中的VERSION中的clusterID 覆盖data/current下的VERSION中的clusterID

-rw-r--r--. 1 root root 30 Nov 29 00:26 edits_0000000000000000014-0000000000000000015
-rw-r--r--. 1 root root 30 Nov 29 00:28 edits_0000000000000000016-0000000000000000017
-rw-r--r--. 1 root root 30 Nov 29 00:30 edits_0000000000000000018-0000000000000000019
-rw-r--r--. 1 root root 30 Nov 29 00:32 edits_0000000000000000020-0000000000000000021
-rw-r--r--. 1 root root 30 Nov 29 00:34 edits_0000000000000000022-0000000000000000023
-rw-r--r--. 1 root root 30 Nov 29 00:36 edits_0000000000000000024-0000000000000000025
-rw-r--r--. 1 root root 30 Nov 29 00:38 edits_0000000000000000026-0000000000000000027
-rw-r--r--. 1 root root 30 Nov 29 00:40 edits_0000000000000000028-0000000000000000029
-rw-r--r--. 1 root root 30 Nov 29 00:42 edits_0000000000000000030-0000000000000000031
-rw-r--r--. 1 root root 30 Nov 29 00:44 edits_0000000000000000032-0000000000000000033
-rw-r--r--. 1 root root 30 Nov 29 00:46 edits_0000000000000000034-0000000000000000035
-rw-r--r--. 1 root root 30 Nov 29 00:48 edits_0000000000000000036-0000000000000000037
-rw-r--r--. 1 root root 30 Nov 29 00:50 edits_0000000000000000038-0000000000000000039
-rw-r--r--. 1 root root 30 Nov 29 00:52 edits_0000000000000000040-0000000000000000041
-rw-r--r--. 1 root root 30 Nov 29 00:54 edits_0000000000000000042-0000000000000000043
-rw-r--r--. 1 root root 1048576 Nov 29 00:54 edits_inprogress_0000000000000000044
-rw-r--r--. 1 root root 196 Nov 29 00:05 fsimage_0000000000000000000
-rw-r--r--. 1 root root 62 Nov 29 00:05 fsimage_0000000000000000000.md5
-rw-r--r--. 1 root root 3 Nov 29 00:54 seen_txid
-rw-r--r--. 1 root root 207 Nov 29 00:05 VERSION
[root@node1 current]# pwd
/opt/hadoop2/dfs/name/current


然后重启hdfs 可以看到没有启动的datanode 已经启动.

出现这个问题的原因是,在第一次格式化hdfs后,启动并使用了hadoop,在使用一段时间之后,又执行了 hdfs namenode -format命令,在这个过程中,出现流程上的错误,导致

namenode的clusterID重新生成了,但是datanode的clusterID没有重新生成

踩