[导入]Eclipse 导入/编译 Hadoop 源码
2016-06-05 15:26
417 查看
http://www.cnblogs.com/errorx/p/3779578.html
eclipse:
Maven:
libprotoc :https://developers.google.com/protocol-buffers/
hadoop:http://www.apache.org/dyn/closer.cgi/hadoop/common/
添加开源中国maven库:http://maven.oschina.net/home.html
maven\conf\settings.xml
<localRepository>path</localRepository>
进入hadoop-maven-plugins文件夹,执行 mvn install
返回源码根目录,执行 mvn eclipse:eclipse –DskipTests
eclipse在任意目录创建新的WorkSpace
eclipse设置Maven:window->preference->maven->{Installations...;user Settings:maven\conf\settings.xml}
eclipse:File->inport->Existing Projects into WorkSpace->Hadoop源码根目录
hadoop-streaming中build path错误->Java Build Path->Source:
删除...hadoop-yarn-server-resourcemanager/conf
Link Source:源码根目录/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf,再随便起个名字;inclusion patterns:capacity-scheduler.xml;exclusion patters:**/*.java
org.apache.hadoop.io.serializer.avro.TestAvroSerialization
下载avro-tools-1.7.4.jar:http://archive.apache.org/dist/avro/avro-1.7.4/java/
进入目录:源码根目录\hadoop-common-project\hadoop-common\src\test\avro
java -jar path/to/avro-tools-1.7.4.jar compile schema avroRecord.avsc ..\java
eclipse 刷新
org.apache.hadoop.ipc.protobuf.TestProtos
进入目录:源码根目录\hadoop-common-project\hadoop-common\src\test\proto
protoc --java_out=../java *.proto
eclipse 刷新
project->clean..->clean all projects & Build the entire workspace
1.准备工作
jdk:eclipse:
Maven:
libprotoc :https://developers.google.com/protocol-buffers/
hadoop:http://www.apache.org/dyn/closer.cgi/hadoop/common/
添加开源中国maven库:http://maven.oschina.net/home.html
maven\conf\settings.xml
<localRepository>path</localRepository>
<localRepository>path</localRepository> <mirrors> <mirror> <id>nexus-osc</id> <mirrorOf>*</mirrorOf> <name>Nexus osc</name> <url>http://maven.oschina.net/content/groups/public/</url> </mirror> </mirrors>
2.导入
将hadoop源码解压到一个目录,注意目录层次不要太深,否则可能无法解压。进入hadoop-maven-plugins文件夹,执行 mvn install
返回源码根目录,执行 mvn eclipse:eclipse –DskipTests
eclipse在任意目录创建新的WorkSpace
eclipse设置Maven:window->preference->maven->{Installations...;user Settings:maven\conf\settings.xml}
eclipse:File->inport->Existing Projects into WorkSpace->Hadoop源码根目录
3.错误处理
maven下载pom失败->重新操作hadoop-streaming中build path错误->Java Build Path->Source:
删除...hadoop-yarn-server-resourcemanager/conf
Link Source:源码根目录/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf,再随便起个名字;inclusion patterns:capacity-scheduler.xml;exclusion patters:**/*.java
org.apache.hadoop.io.serializer.avro.TestAvroSerialization
下载avro-tools-1.7.4.jar:http://archive.apache.org/dist/avro/avro-1.7.4/java/
进入目录:源码根目录\hadoop-common-project\hadoop-common\src\test\avro
java -jar path/to/avro-tools-1.7.4.jar compile schema avroRecord.avsc ..\java
eclipse 刷新
org.apache.hadoop.ipc.protobuf.TestProtos
进入目录:源码根目录\hadoop-common-project\hadoop-common\src\test\proto
protoc --java_out=../java *.proto
eclipse 刷新
project->clean..->clean all projects & Build the entire workspace
相关文章推荐
- JAVA 多线程静态同步函数的锁是class 对象
- java反射机制详解
- Java字符串处理之详解String自带方法
- 修饰类的使用以及使用技巧
- SpringMVC整合fastjson
- Java 8 日期时间 API
- Eclipse过时了吗_图文详解如何让Eclipse媲美IDEA
- Mac环境下JAVA开发环境搭建
- Java多线程编程(第四章)
- Java并发编程-线程局部变量(ThreadLocal)
- 排名前30的eclipse快捷键 java必备
- java数学函数
- JAVA_OA(五)(番外篇):SpringMVC乱码解决(post,get)
- java变量&常量
- 以一个方法为例谈Java的多态
- java获取某月的第一天和最后一天
- java基础之数组
- MyEclipse 7.0 用java代码生成序列号
- Java接口与多态
- java.util.Date 和 java.sql.Date 的精确保存问题