hadoop异常:虚拟机上搭建分布式集群org.apache.hadoop.ipc.Client: Retrying connect to server
2015-09-01 19:40
549 查看
hadoop异常:虚拟机上搭建分布式集群org.apache.hadoop.ipc.Client: Retrying connect to server
问题1:查看hdfs存储的状态
hadoop dfsadmin -report
15/09/01 19:33:20 INFO security.Groups: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=300000
15/09/01 19:33:20 WARN conf.Configuration: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
Configured Capacity: 0 (0 KB)
Present Capacity: 0 (0 KB)
DFS Remaining: 0 (0 KB)
DFS Used: 0 (0 KB)
DFS Used%: ?%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
-------------------------------------------------
Datanodes available: 0 (0 total, 0 dead)
Datanodes available: 0 (0 total, 0 dead)
同时出现问题2:显示没有可用的数据节点。
登陆到数据节点,查看日志
2015-09-01 19:34:19,066 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoopm/192.168.***.*** :9000. Already tried 8
time(s).
原因排查:
1.防火墙的问题(在namenode上关闭防火墙,如果不关闭,也有可能是namenode的防火墙问题)
service iptables stop
2.重新格式化hdfs之后也会出现这样的问题
英文论坛上有人这样回复:You'll probably find that even though the name node starts, itdoesn't have any data nodes and is completely empty.Whenever hadoop creates a new filesystem, itassigns a large random number to it to prevent you from mixingdatanodes from different
filesystems on accident. When you reformatthe name node its FS has one ID, but your data nodes still havechunks of the old FS with a different ID and so will refuse toconnect to the namenode. You need to make sure these are cleaned upbefore reformatting. You
can do it just by deleting the datanode directory, although there's probably a more "official" way todo it.
这两种方法都没有解决我的问题,其实我的问题是第3种:
如果在hadoop的配置文件site-core.xml和mapred-site.xml中用的是主节点的hostname:端口号 的话,在主节点的/etc/hosts文件中,127.0.0.1这一行中,不要写主节点的hostname,不然主节点只会监听本地的hdfs和mapreduce端口,所以其他的结点就连不上了,其实所有的节点都不应该在这一行中写自己的hostname
问题1:查看hdfs存储的状态
hadoop dfsadmin -report
15/09/01 19:33:20 INFO security.Groups: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=300000
15/09/01 19:33:20 WARN conf.Configuration: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
Configured Capacity: 0 (0 KB)
Present Capacity: 0 (0 KB)
DFS Remaining: 0 (0 KB)
DFS Used: 0 (0 KB)
DFS Used%: ?%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
-------------------------------------------------
Datanodes available: 0 (0 total, 0 dead)
Datanodes available: 0 (0 total, 0 dead)
同时出现问题2:显示没有可用的数据节点。
登陆到数据节点,查看日志
2015-09-01 19:34:19,066 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoopm/192.168.***.*** :9000. Already tried 8
time(s).
原因排查:
1.防火墙的问题(在namenode上关闭防火墙,如果不关闭,也有可能是namenode的防火墙问题)
service iptables stop
2.重新格式化hdfs之后也会出现这样的问题
英文论坛上有人这样回复:You'll probably find that even though the name node starts, itdoesn't have any data nodes and is completely empty.Whenever hadoop creates a new filesystem, itassigns a large random number to it to prevent you from mixingdatanodes from different
filesystems on accident. When you reformatthe name node its FS has one ID, but your data nodes still havechunks of the old FS with a different ID and so will refuse toconnect to the namenode. You need to make sure these are cleaned upbefore reformatting. You
can do it just by deleting the datanode directory, although there's probably a more "official" way todo it.
这两种方法都没有解决我的问题,其实我的问题是第3种:
如果在hadoop的配置文件site-core.xml和mapred-site.xml中用的是主节点的hostname:端口号 的话,在主节点的/etc/hosts文件中,127.0.0.1这一行中,不要写主节点的hostname,不然主节点只会监听本地的hdfs和mapreduce端口,所以其他的结点就连不上了,其实所有的节点都不应该在这一行中写自己的hostname
相关文章推荐
- Windows安装Apache后的一些设置
- wamp下Apache2.4.x局域网访问403的解决办法
- 解决Exception: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z等一系列问题
- Ubuntu下Apache多站点虚拟主机配置
- linux -- ubuntu 安装apache后,修改默认路径
- php环境搭建教程(phpstrom+xampp)--如何修改apache端口号
- jetty部署异常:rg.apache.jasper.JasperException: PWC6345: There is an error in invoking ja
- Run a Simple Apache Spark App in CDH 5
- 源码安装apache
- Apache CXF + Spring3 + REST + JSON配置
- linux下安装apache详解
- 工具类apache.commons.lang.StringUtils 使用心得
- apache 并发连接数
- apache漏洞修复
- 在extjs的textfield中输入带有特殊字符的字符串,比如/opt/apache-storm-0.9.4/bin/storm ui &但是&符号却无法传递到后台
- Apache之——配置虚拟目录(监听多个端口)
- apache下设置缓存方法详细介绍
- win Apache 使用 openssl 开启ssl步骤
- Apache配置域名跳转、日志切割、静态缓存、防盗链、访问控制
- Apache之——配置虚拟主机