CentOS6.4编译Hadoop2.2.0

1.安装jdk
不要小看这一步，起先装了个jdk1.8.0，出现如下错误。
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.8.1:jar (module-javadocs) on project hadoop-maven-plugins: MavenReportException: Error while creating archive:

[ERROR] Exit code: 1 - /usr/local/hadoop-2.2.0-src/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/util/Exec.java:47: error: unknown tag: String
想是jdk1.8.0修改了doc的某些内容

在/etc/profile中设置环境变量，添加
#set java environment

export JAVA_HOME=/usr/local/jdk1.7.0

export JRE_HOME=/usr/local/jdk1.7.0/jre

export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar/:$JAVA_HOME/jre/lib:$JAVA_HOME/lib

export PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/jre/bin

2.安装maven3.2.1
不需要特别安装，下载下来看只要把bin加入到环境变量PATH中，同样是修改/etc/profile文件
# set mvn environment

export PATH=$PATH:/usr/local/apache-maven-3.2.1/bin

export MAVEN_HOME=/usr/local/apache-maven-3.2.1
执行mav -version查看安装是否成功

3.安装ant1.9.3
不需要特别安装，下载下来看只要把bin加入到环境变量PATH中，同样是修改/etc/profile文件
# set ant environment

export PATH=$PATH:/usr/local/apache-ant-1.9.3/bin
执行ant -version查看安装是否成功

4.安装cmake2.8.12.2

5.安装findbugs2.0.2
这个要ant安装，只要在findbugs2.0.2的源代码路径下执行
ant
就可以了

6.安装forrest 0.9
要把bin加入到环境变量PATH中，同样是修改/etc/profile文件
# set forrest environment

export PATH=$PATH:/usr/local/apache-forrest-0.9/bin

export FORREST_HOME=$FORREST_HOEM:/usr/local/apache-forrest-0.9

7.安装zlib-devel（可能默认已经安装好了）
yum install zlib-devel

8.安装protobuf 2.5.0

./configure

make

make check

make install

9.修改2.2.0的一个bug
https://issues.apache.org/jira/browse/HADOOP-10110
实际上就是在hadoop-common-project/hadoop-auth/pom.xml中添加了几行代码
在<dependencies></dependencies>之间添加
</dependency>

<dependency>
<groupId>org.mortbay.jetty</groupId>
+      <artifactId>jetty-util</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.mortbay.jetty</groupId>
<artifactId>jetty</artifactId>
<scope>test</scope>
</dependency>

10.还有的修改了maven的配置conf/settings.xml，我照此做了，但是感觉这个并不影响
一是在<mirror> </mirrors>之间添加
<mirror>
<id>nexus-osc</id>

<mirrorOf>*</mirrorOf>

<name>Nexusosc</name>

<url>http://maven.oschina.net/content/groups/public/</url>

</mirror>
二是在<profiles></profiles>之间添加
<profile>

<id>jdk-1.7</id>

<activation>

<jdk>1.7</jdk>

</activation>

<repositories>

<repository>

<id>nexus</id>

<name>local private nexus</name>

<url>http://maven.oschina.net/content/groups/public/</url>

<releases>

<enabled>true</enabled>

</releases>

<snapshots>

<enabled>false</enabled>

</snapshots>

</repository>

</repositories>

<pluginRepositories>

<pluginRepository>

<id>nexus</id>

<name>local private nexus</name>

<url>http://maven.oschina.net/content/groups/public/</url>

<releases>

<enabled>true</enabled>

</releases>

<snapshots>

<enabled>false</enabled>

</snapshots>

</pluginRepository>

</pluginRepositories>

</profile>

以上这些过程我都照做了，但我不确定是否都是必须的，有些如果不安装，编译时会出现错误。

11.进入hadoop-2.2.0-src目录开始编译，输入命令
mvn clean package -Pdist,native -DskipTests -Dtar
我是使用root来编译的，之前在某个博客上说不能使用root编译，但是实践中发现会有permission denied的错误，我不确定错误发生的原因，但我是root的编译的。

以下是所碰到的错误：

1. 2.2.0的源码有个bug，需要patch一下
从
https://issues.apache.org/jira/browse/HADOOP-10110
下载patch文件，放到hadoop-src的根目录下，在此目录下执行
patch -p0 > HADOOP-10110.patch

不patch会碰到如下错误

实际上这个patch就是在/hadoop-src/hadoop-common-project/hadoop-auth/pom.xml中添加了如下代码：
<dependency>

<groupId>org.mortbay.jetty</groupId>

<artifactId>jetty-util</artifactId>

<scope>test</scope>

</dependency>
但是注意这段代码要在标签<dependencies></dependencies>之间，否则会报错。

2.如果不安装protoc，报如下错误
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR] mvn <goals> -rf :hadoop-common

3.此错误
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (create-testdirs) on project hadoop-project: Error executing ant tasks: /usr/local/hadoop-2.2.0-src/hadoop-project/target/antrun/build-main.xml (Permission denied) -> [Help
1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.
分析：
最后有个Permission denied，难道是要 root编译才行？

4. terminal会打印出编译的进度，成功是SUCCEED，失败就FAILURE，之后的全SKIPPED，可以由此定位出自己哪一步失败了。

5.此错误
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.8.1:jar (module-javadocs) on project hadoop-maven-plugins: MavenReportException: Error while creating archive:

[ERROR] Exit code: 1 - /usr/local/hadoop-2.2.0-src/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/util/Exec.java:47: error: unknown tag: String

分析：
看样子是没有识别出String，难道是Java的包没加入？
实际上是Java版本的问题，不能用1.8。