您的位置:首页 > Web前端 > Node.js

阿里云 ECS Hadoop-异常“could only be replicated to 0 nodes, instead of 1” 无法上传文件

2018-02-02 17:58 716 查看
阿里云搭建伪分布式遇到这个问题,正常办法解决不了了的时候,可以一试,步骤如下:

1、/etc/hosts 文件的多余主机名全部注释掉 ,新增加一个你的内网IP 和 hadoop的用户名;

注意:这里是内网地址,而不是阿里云的外网地址,先 ifconfig 查看本机内网ip

[root@cx hadoop]# ifconfig

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
inet 172.19.44.21  netmask 255.255.240.0  broadcast 172.19.47.255
ether 00:16:3e:1c:28:6e  txqueuelen 1000  (Ethernet)
RX packets 675510  bytes 256484775 (244.6 MiB)
RX errors 0  dropped 0  overruns 0  frame 0
TX packets 463354  bytes 323423726 (308.4 MiB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
inet 127.0.0.1  netmask 255.0.0.0
loop  txqueuelen 1  (Local Loopback)
RX packets 16893986  bytes 1619851840 (1.5 GiB)
RX errors 0  drop
4000
ped 0  overruns 0  frame 0
TX packets 16893986  bytes 1619851840 (1.5 GiB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


2、更改 /etc/hosts 文件,设置成内网IP 主机名,多余的注释或者删掉

[root@cx ~]# cat /etc/hosts
#0.0.0.0 cx
172.19.44.21 cx
#127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
#127.0.1.1   localhost
#:::1         localhost localhost.localdomain localhost6 localhost6.localdomain6


3、客户端添加映射,我这里用的是windos远程连接的hdfs ,编辑 C:\Windows\System32\drivers\etc\hosts文件,加入云服务器的外网IP和主机名映射。

# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
#      102.54.94.97     rhino.acme.com          # source server
#       38.25.63.10     x.acme.com              # x client host

# localhost name resolution is handled within DNS itself.
#   127.0.0.1       localhost
#   ::1             localhost
#203.208.46.146 dl.google.com
#203.208.46.146 dl-ssl.google.com
#0.0.0.0 account.jetbrains.com

172.128.120.66 windows10.microdone.cn
106.打码.打码.打码 cx


4、连接hdfs时,设置下配置方式,conf.set(“dfs.client.use.datanode.hostname”, “true”); 设置客户端访问方式使用主机名

public class HdfsTest {
Configuration conf = null;
FileSystem fs = null;

@org.junit.Before
public void conn () throws Exception{

conf =new Configuration(false);
conf.set("fs.defaultFS", "hdfs://cx:8020");
conf.set("dfs.replication", "1");
conf.set("dfs.block.size", "3145728");
conf.set("dfs.client.use.datanode.hostname", "true");
fs = FileSystem.get(conf);
}
}


至此,就可以正确上传文件到hdfs了。
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: 
相关文章推荐