连接hdfs异常protocol message end-group tag did not match expected tag
在本地调试连接hdfs进行写文件操作,连接异常。
java.io.IOException: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.; Host Details : local host is: “Gore-PC/10.108.66.211”; destination host is: “10.108.66.81”:9000;
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.
public static void main(String[] args) throws IOException { System.setProperty("hadoop.home.dir", "d:/hadoop-common-2.2.0-bin/"); // 将本地文件上传到hdfs。 String target = "hdfs://10.94.1.104:9000/home/history/sample.csv"; FileInputStream fis = new FileInputStream(new File("D:\\home\\table.csv"));// 读取本地文件 Configuration config = new Configuration(); FileSystem fs = FileSystem.get(URI.create(target), config); OutputStream os = fs.create(new Path(target)); // copy IOUtils.copyBytes(fis, os, 4096, true); System.out.println("拷贝完成..."); }
问题解决:
1、需要连接HDFS namenode主机,如果连接其他datanode的话,会报Connection refused
2、需使用8020端口,如果使用9000端口,会报Protocol message end-group tag did not match expected tag
3、在定义一个FileSystem变量的时候分布式和单机版的方法是不一样的,单机版使用的是FileSystem类的静态函数
FileSystem hdfs = FileSystem.get(conf)
分布式下需要使用Path来获得
Path dstDir = new Path(hdfsPath);
FileSystem hdfs = dstDir.getFileSystem(conf);
否则会报
Wrong FS: hdfs://hdfs://10.108.66.81:8020/…, expected: file:///
4、需要定义执行HDFS的用户,否则会出现Permission denied
System.setProperty(“HADOOP_USER_NAME”, “hdfs”);
- ResourceMananger启动报错:Protocol message end-group tag did not match expected tag
- iOS 使用ProtocolBuffer 时报 Protocol message end-group tag did not match expected tag.
- Protocol message end-group tag did not match expected tag
- Submitted credentials for token did not match the expected credentials.
- org.springframework.dao.InvalidDataAccessApiUsageException: Parameter value [41] did not match expected type [java.lang.Integer (n/a)];
- spring-data-jpa 报错:Parameter value element [null] did not match expected type [java.lang.String (n/a
- Parameter value [Fri Nov 11 00:00:00 CST 2016] did not match expected type [java.util.Date (未解决 求评论
- Submitted credentials for token[***]did not match the expected credentials
- 最权威正解:Submitted credentials for token [xxx]did not match the expected credentials.
- 无法使用SQL login去登陆SQL Server - 'Password did not match'
- Tomcat 7下SSL配置出现异常: '/root/.keystore' did not find a matching property
- The declared package..does not match the expected package
- Navicat 远程连接docker容器中的mysql 报错1251 - Client does not support authentication protocol 解决办法
- Device chain in Chain Description File does not match physical device chain expected 2 device(s) but found 1 device(s)
- 异常:According to TLD tag beanwrite must be empty but is not
- Cygwin apt-cyg ”MD5 sum did not match”
- 【Python】异常:IndentationError: unindent does not match any outer indentation level
- SFTP连接异常 Could not connect to SFTP server. Caused by: com.jcraft.jsch.JSchException: Auth fail
- git checkout 出错原因 (error: pathspec 'master' did not match any file(s) known to )
- python3.5+django1.10:selenium.common.exceptions.WebDriverException->moveto did not match a known com