hadoop集群启动时DataNode节点启动失败
2018-03-27 11:22
549 查看
这一批博文是博主由博客园搬移过来的,所以时间上可能存在混乱,希望大家见谅!
错误日志如下:
解决方法:根据日志进入路径(/usr/java/data/dfs/data)。此时,我们应该能看见其下有/data和/name两个目录。将name/current下的VERSION中的clusterID复制到data/current下的VERSION中,覆盖掉原来的clusterID,让两个保持一致;或者,直接删除掉data和name两个目录下的内容,然后重启。
注意:在重启之前,一定要结束掉hadoop所有进程(sbin/stop-all.sh),否则会出现其他错误。
现在,我们重启之后输入jps就可以看到以下进程了:
以上就是博主为大家介绍的这一板块的主要内容,这都是博主自己的学习过程,希望能给大家带来一定的指导作用,有用的还望大家点个支持,如果对你没用也望包涵,有错误烦请指出。如有期待可关注博主以第一时间获取更新哦,谢谢!
版权声明:本文为博主原创文章,未经博主允许不得转载。
错误日志如下:
************************************************************/ 2018-03-07 18:57:35,121 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT] 2018-03-07 18:57:35,296 WARN org.apache.hadoop.hdfs.server.common.Util: Path /usr/java/data/dfs/data should be specified as a URI in configuration files. Please update hdfs configuration. 2018-03-07 18:57:36,059 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2018-03-07 18:57:36,153 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2018-03-07 18:57:36,153 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started 2018-03-07 18:57:36,155 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is Centpy 2018-03-07 18:57:36,208 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened streaming server at /0.0.0.0:50010 2018-03-07 18:57:36,213 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 1048576 bytes/s 2018-03-07 18:57:36,264 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2018-03-07 18:57:36,296 INFO org.apache.hadoop.http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2018-03-07 18:57:36,298 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode 2018-03-07 18:57:36,298 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2018-03-07 18:57:36,298 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2018-03-07 18:57:36,300 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened info server at 0.0.0.0:50075 2018-03-07 18:57:36,302 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dfs.webhdfs.enabled = false 2018-03-07 18:57:36,302 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50075 2018-03-07 18:57:36,302 INFO org.mortbay.log: jetty-6.1.26 2018-03-07 18:57:36,592 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50075 2018-03-07 18:57:36,942 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 50020 2018-03-07 18:57:36,962 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:50020 2018-03-07 18:57:36,971 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null 2018-03-07 18:57:36,985 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default> 2018 b1ed -03-07 18:57:36,988 WARN org.apache.hadoop.hdfs.server.common.Util: Path /usr/java/data/dfs/data should be specified as a URI in configuration files. Please update hdfs configuration. 2018-03-07 18:57:36,996 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool <registering> (storage id unknown) service to Centpy/192.168.86.127:9000 starting to offer service 2018-03-07 18:57:37,000 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2018-03-07 18:57:37,003 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting 2018-03-07 18:57:37,305 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /usr/java/data/dfs/data/in_use.lock acquired by nodename 7541@Centpy 2018-03-07 18:57:37,306 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for block pool Block pool BP-7509759-192.168.86.127-1520419708605 (storage id ) service to Centpy/192.168.86.127:9000 org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /usr/java/data/dfs/data is in an inconsistent state: node type is incompatible with others. at org.apache.hadoop.hdfs.server.common.Storage.setStorageType(Storage.java:1051) at org.apache.hadoop.hdfs.server.datanode.DataStorage.setFieldsFromProperties(DataStorage.java:304) at org.apache.hadoop.hdfs.server.common.Storage.readProperties(Storage.java:921) at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:372) at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:191) at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:219) at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:837) at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:808) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:280) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:222) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:664) at java.lang.Thread.run(Thread.java:745) 2018-03-07 18:57:37,307 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool BP-7509759-192.168.86.127-1520419708605 (storage id ) service to Centpy/192.168.86.127:9000 2018-03-07 18:57:37,313 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool BP-7509759-192.168.86.127-1520419708605 (storage id ) 2018-03-07 18:57:39,313 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode 2018-03-07 18:57:39,315 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0 2018-03-07 18:57:39,322 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down DataNode at Centpy/192.168.86.127 ************************************************************/日志说明了datanode的clusterID 和 namenode的clusterID 不匹配。
解决方法:根据日志进入路径(/usr/java/data/dfs/data)。此时,我们应该能看见其下有/data和/name两个目录。将name/current下的VERSION中的clusterID复制到data/current下的VERSION中,覆盖掉原来的clusterID,让两个保持一致;或者,直接删除掉data和name两个目录下的内容,然后重启。
注意:在重启之前,一定要结束掉hadoop所有进程(sbin/stop-all.sh),否则会出现其他错误。
现在,我们重启之后输入jps就可以看到以下进程了:
8862 NameNode 8982 DataNode 9131 SecondaryNameNode 9506 Jps 9374 NodeManager 9269 ResourceManager
以上就是博主为大家介绍的这一板块的主要内容,这都是博主自己的学习过程,希望能给大家带来一定的指导作用,有用的还望大家点个支持,如果对你没用也望包涵,有错误烦请指出。如有期待可关注博主以第一时间获取更新哦,谢谢!
版权声明:本文为博主原创文章,未经博主允许不得转载。
相关文章推荐
- hadoop集群启动时DataNode节点启动失败
- hadoop集群运行jps命令以后Datanode节点未启动的解决办法
- 启动 Hadoop 集群时datanode节点不启动原因之一——格式化次数太多的解决办法
- hadoop集群启动之后dataNode节点没有启动
- hadoop集群运行jps命令以后Datanode节点未启动的解决办法
- 当Hadoop 启动节点Datanode失败解决
- hadoop集群启动后,datanode节点上jps后NodeManager无法启动
- hadoop2.5.x搭建集群启动jps在datanode节点无法显示datanode
- hadoop集群运行jps命令以后Datanode节点未启动的解决办法
- Hadoop集群启动之后,datanode节点未正常启动的问题
- Hadoop 启动节点Datanode失败解决
- 在HDFS集群中优化secondary namenode到datanode1节点上,并做重启hdfs集群后,datanode1启动失败
- Hadoop 启动节点Datanode失败解决
- Hadoop datanode正常启动,但是Live nodes中却缺少节点的问题
- hadoop学习笔记(2)-hadoop安装目录权限的问题导致datanode启动失败
- Hadoop集群所有的DataNode都启动不了解决办法
- 当一个节点在CM中出现2条记录,导致启动hadoop集群失败,host_id 始终找不到匹配的host_identifier
- hadoop集群配置datanode无法启动的原因
- 集群datanode节点失败导致hdfs写失败
- Hadoop集群搭建过程中DataNode与TaskTracker节点的问题