您的位置:首页 > 编程语言 > Java开发

在cygwin上用ant编译hadoop的eclipse插件

2013-12-24 16:17 423 查看

1.参考了http://bigboy.iteye.com/blog/1766382,和其不同的地方为:

<!--
<copyfile="${hadoop.root}/build/hadoop-core-${version}.jar"tofile="${build.dir}/lib/hadoop-core.jar"verbose="true"/>
<copyfile="${hadoop.root}/build/ivy/lib/Hadoop/common/commons-cli-${commons-cli.version}.jar"  todir="${build.dir}/lib"verbose="true"/>
-->
<copyfile="${hadoop.root}/build/hadoop-core-${version}.jar"tofile="${build.dir}/lib/hadoop-core.jar"verbose="true"/>
<copyfile="${hadoop.root}/build/ivy/lib/Hadoop/common/commons-cli-1.2.jar"  todir="${build.dir}/lib"verbose="true"/>
<copyfile="${hadoop.root}/lib/commons-configuration-1.6.jar"todir="${build.dir}/lib" verbose="true"/>
<copyfile="${hadoop.root}/lib/commons-httpclient-3.0.1.jar"todir="${build.dir}/lib" verbose="true"/>
<copyfile="${hadoop.root}/lib/commons-lang-2.4.jar"todir="${build.dir}/lib" verbose="true"/>
<copyfile="${hadoop.root}/lib/jackson-core-asl-1.0.1.jar"todir="${build.dir}/lib" verbose="true"/>
<copyfile="${hadoop.root}/lib/jackson-mapper-asl-1.0.1.jar"todir="${build.dir}/lib" verbose="true"/>


2.最后在%hadoop_root%src\contrib\eclipse-plugin下ant jar报错:软件包org.apache.hadoop.fs不存在

参考http://blog.csdn.net/yundixiaoduo/article/details/7451753

3.ant -v jar提示找不到类org.eclipse.jdt.debug.ui.launchConfigurations.JavaApplicationLaunchShortcut

发现是因为自己用的eclipse太老了,在plugins文件夹中找到的jar包为org.eclipse.jdt.debug.ui_3.2.100.v20070531-1800.jar 2007年的jar包中没有这个类,遂随便下了org.eclipse.jdt.debug.ui_3.6.0.v20110512.jar,包中有此类,放在ant/lib下不再提示找不到此类

4.ant -v jar报错:

[javac] C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\src\java\org\apache\hadoop\eclipse\launch\HadoopApplicationLaunchShortcut.java:48:错误: 无法访问ILaunchShortcut2
[javac] public class HadoopApplicationLaunchShortcut extends
[javac]        ^
[javac]   找不到org.eclipse.debug.ui.ILaunchShortcut2的类文件
[javac]C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\src\java\org\apache\hadoop\eclipse\launch\HadoopApplicationLaunchShortcut.java:66:错误: 找不到符号
[javac]        super.findLaunchConfiguration(type, configType);
[javac]         ^
[javac]   符号:   变量 super
[javac]   位置: 类HadoopApplicationLaunchShortcut
[javac]C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\src\java\org\apache\hadoop\eclipse\launch\HadoopApplicationLaunchShortcut.java:67:错误: 找不到符号
[javac]     if (iConf == null)iConf = super.createConfiguration(type);
[javac]                               ^
[javac]   符号:   变量 super
[javac]   位置: 类 HadoopApplicationLaunchShortcut
[javac]C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\src\java\org\apache\hadoop\eclipse\launch\HadoopApplicationLaunchShortcut.java:60:错误: 方法不会覆盖或实现超类型的方法
[javac]   @Override
[javac]   ^
[javac] 注:C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\src\java\org\apache\hadoop\eclipse\server\JarModule.java使用或覆盖了已过时的API。
[javac] 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
[javac] 注: 某些输入文件使用了未经检查或不安全的操作。
[javac] 注: 有关详细信息, 请使用 -Xlint:unchecked 重新编译。
[javac] 4 个错误

BUILD FAILED


服了,彻底投降!!

5.在网上随便找了hadoop-eclipse-plugin-1.0.4.jar放在eclipse/plugins/下重启eclipse,在showview>other中找到Map/Reduce Locations 配置host:127.0.0.1 Port:9000 9001

最终,eclipse中显示了hdfs的文件列表

6.总结:

1)第一次使用ant对其编译原理和结构不熟,一头雾水的瞎搜,瞎搞。。。

2)对xml认识不足,只能照着别人的瞎改。。。

build.xml的细致分析参见:http://blog.sina.com.cn/s/blog_4ed8b87701011c8h.html

7.最终原因

最终原因为装的eclipse版本太低(3.3)07年发布的版本,在网上查到说要3.3以上的版本才能顺利的编译hadoop的eclipse插件,否则会出现各种找不到jar包,就是上面我遇到的各种问题,最终下了最新Kepler Service Release 1,64位的,后eclipse无法打开:
Failed to load the JNI shared library
原因是原来的eclipse是32位的,jdk也是32位的,现在是64位的eclipse 32位的jdk,不匹配。卸载32位的jdk,下载64位的jdk,重新配置环境变量,问题解决。

在cygwin中 cd /hadoophome/src/contrib/eclipse_plugins ant jar 一切顺利,在hadoop-1.0.0\build\contrib\eclipse-plugin\下生成了hadoop-eclipse-plugin-1.0.0.jar
E5430 ~/hadoop-1.0.0/src/contrib/eclipse-plugin
$ ant jar
Buildfile: C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\build.xml

check-contrib:

init:
[echo] contrib: eclipse-plugin

init-contrib:

ivy-download:
[get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar [get] To: C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\ivy\ivy-2.1.0.jar
[get] Not modified - so not downloaded

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:
[ivy:configure] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\ivy\ivysettings.xml

ivy-resolve-common:
[ivy:resolve] :: resolving dependencies :: org.apache.hadoop#eclipse-plugin;working@ZhangWenL-E5430
[ivy:resolve]   confs: [common]
[ivy:resolve]   found commons-logging#commons-logging;1.0.4 in maven2
[ivy:resolve]   found log4j#log4j;1.2.15 in maven2
[ivy:resolve] :: resolution report :: resolve 209ms :: artifacts dl 5ms
---------------------------------------------------------------------
|                  |            modules            ||   artifacts   |
|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
|      common      |   2   |   0   |   0   |   0   ||   2   |   0   |
---------------------------------------------------------------------

ivy-retrieve-common:
[ivy:retrieve] :: retrieving :: org.apache.hadoop#eclipse-plugin [sync]
[ivy:retrieve]  confs: [common]
[ivy:retrieve]  0 artifacts copied, 2 already retrieved (0kB/8ms)
[ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead
[ivy:cachepath] :: loading settings :: file = C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\ivy\ivysettings.xml

compile:
[echo] contrib: eclipse-plugin
[javac] C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\src\contrib\eclipse-plugin\build.xml:76: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 45 source files to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\classes
[javac] 注: 某些输入文件使用或覆盖了已过时的 API。
[javac] 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
[javac] 注: 某些输入文件使用了未经检查或不安全的操作。
[javac] 注: 有关详细信息, 请使用 -Xlint:unchecked 重新编译。

jar:
[copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
[copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\hadoop-core-1.0.0.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\hadoop-core.jar
[copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
[copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\ivy\lib\Hadoop\common\commons-cli-1.2.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\commons-cli-1.2.jar
[copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
[copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\lib\commons-configuration-1.6.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\commons-configuration-1.6.jar
[copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
[copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\lib\commons-httpclient-3.0.1.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\commons-httpclient-3.0.1.jar
[copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
[copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\lib\commons-lang-2.4.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\commons-lang-2.4.jar
[copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
[copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\lib\jackson-core-asl-1.0.1.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\jackson-core-asl-1.0.1.jar
[copy] Copying 1 file to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib
[copy] Copying C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\lib\jackson-mapper-asl-1.0.1.jar to C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\lib\jackson-mapper-asl-1.0.1.jar
[jar] Building jar: C:\cygwin64\home\i-zhangwenlong\hadoop-1.0.0\build\contrib\eclipse-plugin\hadoop-eclipse-plugin-1.0.0.jar

BUILD SUCCESSFUL
Total time: 7 seconds
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: