hadoop windows下调试报错的分析org.apache.hadoop.io.nativeio.NativeIO$Windows
2016-03-24 17:14
537 查看
升级了服务器上hadoop版本到2.7.2,原来是2.4.1,响应的修改了pom.xml的hadoop版本。
本地运行时出现如下错误:
java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.createDirectoryWithMode0(Ljava/lang/String;I)V
at org.apache.hadoop.io.nativeio.NativeIO$Windows.createDirectoryWithMode0(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO$Windows.createDirectoryWithMode(NativeIO.java:524)
at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:476)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:530)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:507)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:305)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:133)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:144)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at com.cloud.bigdata.ad.background.TaskExecutor.run(TaskExecutor.java:135)
at com.cloud.bigdata.ad.test.background.executors.isp.HostPVByISPHostTaskExecutorTest.test(HostPVByISPHostTaskExecutorTest.java:28)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at org.springframework.test.ConditionalTestCase.runBare(ConditionalTestCase.java:79)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
分析结果是由于本地的hadoop是windows下编译的2.4.1版本,代码用了hadoop2.7.2的jar造成的。
解决方法(亲测)
重新下载hadoop2.7.2在windows下编译,本人在win7 64位下编译。编译配置有不少陷阱花了不少时间。
编译后将hadoop目录覆盖原来的%HADOOP_HOME%环境变量,同时注意PATH中应有%HADOOP_HOME%/bin,因为windows本地允许需要新编译出来的hadoop.dll,PATH需要能找到。
重新运行程序,程序正常执行完成。
本地运行时出现如下错误:
java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.createDirectoryWithMode0(Ljava/lang/String;I)V
at org.apache.hadoop.io.nativeio.NativeIO$Windows.createDirectoryWithMode0(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO$Windows.createDirectoryWithMode(NativeIO.java:524)
at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:476)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:530)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:507)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:305)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:133)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:144)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at com.cloud.bigdata.ad.background.TaskExecutor.run(TaskExecutor.java:135)
at com.cloud.bigdata.ad.test.background.executors.isp.HostPVByISPHostTaskExecutorTest.test(HostPVByISPHostTaskExecutorTest.java:28)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at org.springframework.test.ConditionalTestCase.runBare(ConditionalTestCase.java:79)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
分析结果是由于本地的hadoop是windows下编译的2.4.1版本,代码用了hadoop2.7.2的jar造成的。
解决方法(亲测)
重新下载hadoop2.7.2在windows下编译,本人在win7 64位下编译。编译配置有不少陷阱花了不少时间。
编译后将hadoop目录覆盖原来的%HADOOP_HOME%环境变量,同时注意PATH中应有%HADOOP_HOME%/bin,因为windows本地允许需要新编译出来的hadoop.dll,PATH需要能找到。
重新运行程序,程序正常执行完成。
相关文章推荐
- Apache Maven 入门篇(下)
- Apache Maven 入门篇 ( 上 )
- Centos 6.6 在LAMP环境 Apache配置vhost
- Apache配置基于域名的虚拟主机
- Cannot load php5apache2_4.dll into server解决办法;
- wamp安装后,apache访问出现403
- 编译安装apache2.4.18遇到的问题以及解决办法
- Apache 创建虚拟主机目录和设置默认访问页面
- 持续集成篇_03_svn版本管理系统的使用(centos+subversion+apache+jsvnadmin+tortoisesvn)
- org.apache.commons.lang.StringUtils(StringUtils工具类的常用方法)
- org.apache.ibatis.binding.BindingException: Invalid bound statement (not found)错误解决方法
- linux(centos7)+apache+php7 安装教程
- Exception in thread "scheduler_Worker-2" java.lang.NoClassDefFoundError: org/apache/log4j/spi/Throwa
- 【志银】php5.6-Apache2.4-mysql5.6环境配置(win7_64位)
- Windows Server里80端口与多IP绑定(可实现IIS与Apache共享80端口)
- java.lang.NoSuchMethodException: org.apache.catalina.deploy.WebXml addServlet异常
- apache的server-status如何分析的技术说明
- 年终福利,PHP7+Apache2.4+MySQL5.6 源码编译安装,环境配置,搭建你自己的LAMP环境
- org.apache.hadoop.fs.ChecksumException: Checksum error
- apache-Jmeter简单使用教程