hadoop Safe mode is ON 的解决办法
2016-03-19 22:30
295 查看
搭了一个hadoop集群环境,近期总是出现读写文件错误的情况,查看name node的日志显示 (Safe
mode is ON)
Safe mode is ON. The ratio of reported blocks 0.2000 has not reached the threshold 0.9990. Safe mode will be turned off automatically.
再往上翻
org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /home/hadoop/tmp/mapred/system. Name node is in safe mode.
google 解决方案有三种
1)执行命令:bin/hadoop dfsadmin -safemode leave
dfsadmin -safemode value 参数value的说明如下:
enter - 进入安全模式
leave - 强制NameNode离开安全模式
get - 返回安全模式是否开启的信息
wait - 等待安全模式结束。
2)重新格式化文件系统 =。=
3)bin/hadoop fsck /
第一种方法,需要每次都执行一遍,很纠结~
第二种方法,format的意思你懂得。。。。
第三种方法,如果数据多,那执行起来会很慢,没办法,慢慢等吧。
hadoop@mhadoop:~/hadoop-1.0.0$ bin/hadoop fsck /
FSCK started by hadoop from /192.168.1.110 for path / at Wed Mar 07 17:57:43 CST 2012
.
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0001/job.jar: CORRUPT block blk_-8710406465335110666
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0001/job.jar: MISSING 1 blocks of total size 142465 B..
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0002/job.jar: CORRUPT block blk_8343735010139595189
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0002/job.jar: MISSING 1 blocks of total size 142465 B..
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0004/job.jar: CORRUPT block blk_4796918520991551976
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0004/job.jar: MISSING 1 blocks of total size 142465 B..
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0005/job.jar: CORRUPT block blk_8732649167016211914
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0005/job.jar: MISSING 1 blocks of total size 142465 B..
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0006/job.jar: CORRUPT block blk_2190395249671123528
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0006/job.jar: MISSING 1 blocks of total size 142465 B..
/tmp/hadoop/tmp/mapred/system/jobtracker.info: Under replicated blk_3010497672022840180_1511. Target Replicas is 2 but found 1 replica(s).
.
/user/hadoop/input/README.txt: Under replicated blk_1278803782511847062_1510. Target Replicas is 2 but found 1 replica(s).
..
/user/hadoop/output/_logs/history/job_201202290728_0007_1330473591943_hadoop_word+count: CORRUPT block blk_-165491155081655193
/user/hadoop/output/_logs/history/job_201202290728_0007_1330473591943_hadoop_word+count: MISSING 1 blocks of total size 13581 B..
/user/hadoop/output/_logs/history/job_201202290728_0007_conf.xml: CORRUPT block blk_359906738089887519
/user/hadoop/output/_logs/history/job_201202290728_0007_conf.xml: MISSING 1 blocks of total size 20344 B..
/user/hadoop/output/part-r-00000: CORRUPT block blk_-3883649133311907523
/user/hadoop/output/part-r-00000: MISSING 1 blocks of total size 158081 B.Status: CORRUPT
Total size: 905701 B
Total dirs: 20
Total files: 11
Total blocks (validated): 10 (avg. block size 90570 B)
********************************
CORRUPT FILES: 8
MISSING BLOCKS: 8
MISSING SIZE: 904331 B
CORRUPT BLOCKS: 8
********************************
Minimally replicated blocks: 2 (20.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 2 (20.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 2
Average block replication: 0.2
Corrupt blocks: 8
Missing replicas: 2 (100.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Wed Mar 07 17:57:43 CST 2012 in 11 milliseconds
The filesystem under path '/' is CORRUPT
mode is ON)
Safe mode is ON. The ratio of reported blocks 0.2000 has not reached the threshold 0.9990. Safe mode will be turned off automatically.
再往上翻
org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /home/hadoop/tmp/mapred/system. Name node is in safe mode.
google 解决方案有三种
1)执行命令:bin/hadoop dfsadmin -safemode leave
dfsadmin -safemode value 参数value的说明如下:
enter - 进入安全模式
leave - 强制NameNode离开安全模式
get - 返回安全模式是否开启的信息
wait - 等待安全模式结束。
2)重新格式化文件系统 =。=
3)bin/hadoop fsck /
第一种方法,需要每次都执行一遍,很纠结~
第二种方法,format的意思你懂得。。。。
第三种方法,如果数据多,那执行起来会很慢,没办法,慢慢等吧。
hadoop@mhadoop:~/hadoop-1.0.0$ bin/hadoop fsck /
FSCK started by hadoop from /192.168.1.110 for path / at Wed Mar 07 17:57:43 CST 2012
.
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0001/job.jar: CORRUPT block blk_-8710406465335110666
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0001/job.jar: MISSING 1 blocks of total size 142465 B..
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0002/job.jar: CORRUPT block blk_8343735010139595189
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0002/job.jar: MISSING 1 blocks of total size 142465 B..
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0004/job.jar: CORRUPT block blk_4796918520991551976
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0004/job.jar: MISSING 1 blocks of total size 142465 B..
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0005/job.jar: CORRUPT block blk_8732649167016211914
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0005/job.jar: MISSING 1 blocks of total size 142465 B..
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0006/job.jar: CORRUPT block blk_2190395249671123528
/tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0006/job.jar: MISSING 1 blocks of total size 142465 B..
/tmp/hadoop/tmp/mapred/system/jobtracker.info: Under replicated blk_3010497672022840180_1511. Target Replicas is 2 but found 1 replica(s).
.
/user/hadoop/input/README.txt: Under replicated blk_1278803782511847062_1510. Target Replicas is 2 but found 1 replica(s).
..
/user/hadoop/output/_logs/history/job_201202290728_0007_1330473591943_hadoop_word+count: CORRUPT block blk_-165491155081655193
/user/hadoop/output/_logs/history/job_201202290728_0007_1330473591943_hadoop_word+count: MISSING 1 blocks of total size 13581 B..
/user/hadoop/output/_logs/history/job_201202290728_0007_conf.xml: CORRUPT block blk_359906738089887519
/user/hadoop/output/_logs/history/job_201202290728_0007_conf.xml: MISSING 1 blocks of total size 20344 B..
/user/hadoop/output/part-r-00000: CORRUPT block blk_-3883649133311907523
/user/hadoop/output/part-r-00000: MISSING 1 blocks of total size 158081 B.Status: CORRUPT
Total size: 905701 B
Total dirs: 20
Total files: 11
Total blocks (validated): 10 (avg. block size 90570 B)
********************************
CORRUPT FILES: 8
MISSING BLOCKS: 8
MISSING SIZE: 904331 B
CORRUPT BLOCKS: 8
********************************
Minimally replicated blocks: 2 (20.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 2 (20.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 2
Average block replication: 0.2
Corrupt blocks: 8
Missing replicas: 2 (100.0 %)
Number of data-nodes: 1
Number of racks: 1
FSCK ended at Wed Mar 07 17:57:43 CST 2012 in 11 milliseconds
The filesystem under path '/' is CORRUPT
相关文章推荐
- 详解HDFS Short Circuit Local Reads
- Hadoop_2.1.0 MapReduce序列图
- 使用Hadoop搭建现代电信企业架构
- 单机版搭建Hadoop环境图文教程详解
- hadoop常见错误以及处理方法详解
- hadoop 单机安装配置教程
- hadoop的hdfs文件操作实现上传文件到hdfs
- hadoop实现grep示例分享
- Apache Hadoop版本详解
- linux下搭建hadoop环境步骤分享
- hadoop client与datanode的通信协议分析
- hadoop中一些常用的命令介绍
- Hadoop单机版和全分布式(集群)安装
- 用PHP和Shell写Hadoop的MapReduce程序
- hadoop map-reduce中的文件并发操作
- Hadoop1.2中配置伪分布式的实例
- java结合HADOOP集群文件上传下载
- 让python在hadoop上跑起来
- 用python + hadoop streaming 分布式编程(一) -- 原理介绍,样例程序与本地调试
- Hadoop安装感悟