Hadoop YARN ERROR 1/1 local-dirs are bad *， 1/1 log-dirs are bad *

 

Hadoop YARN ERROR 1/1 local-dirs are bad *， 1/1 log-dirs are bad *

标签： hadoopyarnnodemanager
2015-07-16 16:08 3800人阅读 评论(0) 收藏 举报


 分类：

hadoop（3） 


版权声明：本文为博主原创文章，未经博主允许不得转载。





查看 nodemanager 日志发下 如下信息



2015-07-16 15:28:58,643 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/data/yarn/nmlocal error, used space above threshold of 90.0%, removing from
list of valid directories
2015-07-16 15:28:58,645 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/logs/yarn/nmlogs error, used space above threshold of 90.0%, removing from
list of valid directories
2015-07-16 15:28:58,645 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) failed: 1/1 local-dirs are bad: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are bad:
/opt/beh/logs/yarn/nmlogs
2015-07-16 15:28:58,645 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 1/1 local-dirs are bad: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs
are bad: /opt/beh/logs/yarn/nmlogs
2015-07-16 15:28:58,648 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType
for class org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService
2015-07-16 15:28:58,648 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: per directory file limit = 8192
2015-07-16 15:28:58,658 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType
for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker
2015-07-16 15:28:58,711 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class
class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.

解决方法 
 删除部分没用的数据  让磁盘已用容量下降到90%一下



2015-07-16 15:29:06,808 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Notifying ContainerManager to unblock new container-requests
2015-07-16 15:38:58,817 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/data/yarn/nmlocal passed disk check, adding to list of valid directories.
2015-07-16 15:38:58,821 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/logs/yarn/nmlogs passed disk check, adding to list of valid directories.
2015-07-16 15:38:58,821 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) turned good: 1/1 local-dirs are good: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs
are good: /opt/beh/logs/yarn/nmlogs