您的位置:首页 > Web前端 > Node.js

hadoop datanode重启出现namenode的clusterID和datanode的不一致。

2016-05-10 15:35 537 查看
日志报出的错误是这样的:

016-05-10 15:14:05,620 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for block pool Block pool <registering> (storage id DS-6116241-10.20.18.160-50010-1462417536805) service to cdhhadoop1/10.20.18.158:8020
java.io.IOException: Incompatible clusterIDs in /data/1/dfs/dn: namenode clusterID = CID-dc4b2cf0-4676-4517-ab93-dacbcde16e04; datanode clusterID = CID-ac9d3097-75ad-45b5-8af3-b0e930400956
at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:407)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:191)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:219)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:916)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:887)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:311)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:218)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:660)
at java.lang.Thread.run(Thread.java:745)
2016-05-10 15:14:05,622 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (storage id DS-6116241-10.20.18.160-50010-1462417536805) service to cdhhadoop1/10.20.18.158:8020
2016-05-10 15:14:05,624 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool ID needed, but service not yet registered with NN
java.lang.Exception: trace
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.getBlockPoolId(BPOfferService.java:155)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.remove(BlockPoolManager.java:91)
at org.apache.hadoop.hdfs.server.datanode.DataNode.shutdownBlockPool(DataNode.java:846)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.shutdownActor(BPOfferService.java:387)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.cleanUp(BPServiceActor.java:487)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:683)
at java.lang.Thread.run(Thread.java:745)
2016-05-10 15:14:05,624 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool <registering> (storage id DS-6116241-10.20.18.160-50010-1462417536805)
2016-05-10 15:14:05,624 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool ID needed, but service not yet registered with NN
java.lang.Exception: trace
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.getBlockPoolId(BPOfferService.java:155)
at org.apache.hadoop.hdfs.server.datanode.DataNode.shutdownBlockPool(DataNode.java:848)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.shutdownActor(BPOfferService.java:387)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.cleanUp(BPServiceActor.java:487)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:683)
at java.lang.Thread.run(Thread.java:745)
2016-05-10 15:14:07,624 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode
2016-05-10 15:14:07,626 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0
2016-05-10 15:14:07,628 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at cdhhadoop3/10.20.18.160
************************************************************/


解决方法:
首先要确认你的数据文件夹是正确的。文件夹权限是正确的。

然后删除datanode里面的数据,不要删除文件夹。删除里面的文件就行了。

这样重启datanode的时候会重新去生成文件。

引起这个问题,可能是重启namenode或者format namenode的时候,datanode是死的。所以引起datanode和namenode上面的数据不一致。
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: