Hadoop YARN ERROR 1/1 local-dirs are bad *, 1/1 log-dirs are bad *
2015-07-16 16:08
393 查看
查看 nodemanager 日志发下 如下信息
2015-07-16 15:28:58,643 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/data/yarn/nmlocal error, used space above threshold of 90.0%, removing from list of valid directories
2015-07-16 15:28:58,645 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/logs/yarn/nmlogs error, used space above threshold of 90.0%, removing from list of valid directories
2015-07-16 15:28:58,645 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) failed: 1/1 local-dirs are bad: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are bad: /opt/beh/logs/yarn/nmlogs
2015-07-16 15:28:58,645 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 1/1 local-dirs are bad: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are bad: /opt/beh/logs/yarn/nmlogs
2015-07-16 15:28:58,648 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService
2015-07-16 15:28:58,648 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: per directory file limit = 8192
2015-07-16 15:28:58,658 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker
2015-07-16 15:28:58,711 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'.
Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
解决方法
删除部分没用的数据 让磁盘已用容量下降到90%一下
2015-07-16 15:29:06,808 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Notifying ContainerManager to unblock new container-requests
2015-07-16 15:38:58,817 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/data/yarn/nmlocal passed disk check, adding to list of valid directories.
2015-07-16 15:38:58,821 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/logs/yarn/nmlogs passed disk check, adding to list of valid directories.
2015-07-16 15:38:58,821 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) turned good: 1/1 local-dirs are good: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are good: /opt/beh/logs/yarn/nmlogs
相关文章推荐
- 详解HDFS Short Circuit Local Reads
- Hadoop_2.1.0 MapReduce序列图
- 使用Hadoop搭建现代电信企业架构
- 单机版搭建Hadoop环境图文教程详解
- hadoop常见错误以及处理方法详解
- hadoop 单机安装配置教程
- hadoop的hdfs文件操作实现上传文件到hdfs
- hadoop实现grep示例分享
- Apache Hadoop版本详解
- linux下搭建hadoop环境步骤分享
- hadoop client与datanode的通信协议分析
- Hadoop2.X/YARN环境搭建--CentOS7.0 JDK配置
- hadoop中一些常用的命令介绍
- Hadoop单机版和全分布式(集群)安装
- 用PHP和Shell写Hadoop的MapReduce程序
- hadoop map-reduce中的文件并发操作
- Hadoop1.2中配置伪分布式的实例
- Hadoop2.X/YARN环境搭建--CentOS7.0系统配置
- linux开机启动nodemanager步骤
- java结合HADOOP集群文件上传下载