Hadoop2.0.4(YARN)源码编译

在Ubuntu12.04平台编译hadoop-2.0.4-alpha-src

首先去http://www.apache.org/dyn/closer.cgi/hadoop/common/选择合适站点下载hadoop-2.0.4-alpha-src源码包

根据src/BUILDING.TXT描述

hadoop-2.0.4-alpha-src

Requirements:

* Unix System

* JDK 1.6

* Maven 3.0

* Findbugs 1.3.9 (if running findbugs)

* ProtocolBuffer 2.4.1+ (for MapReduce and HDFS)

* CMake 2.6 or newer (if compiling native code)

* Internet connection for first build (to fetch all Maven and Hadoop dependencies)

其中Maven 3.0安装，可以使用如下shell命令：

sudo add-apt-repository ppa:natecarlson/maven3

sudo apt-get update && sudo apt-get install maven3

sudo ln -s /usr/bin/mvn3 /usr/bin/mvn

CMake同样可一可以sudo apt-get install cmake

如果没安装CMake，运行

mvn package -Pdist,native,docs -DskipTests -Dtar

会报错,mvn中无生命周期dist,native....

ProtocolBuffer 2.4.1

如果没安装ProtocolBuffer，

mvn package -Pdist,native,docs -DskipTests -Dtar

会出现如下错误：

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit

ch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please rea

d the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionE
xception

[ERROR]

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :hadoop-hdfs

 

解决方案参照http://wiki.apache.org/hadoop/HowToContribute安装ProtocolBuffer

如果问题继续，则在hadoop-2.0.4-alpha-src目录下

grep 'com.google.protobuf' -R --include=*.pom .

确定对应文件中

<dependency>

        <groupId>com.google.protobuf</groupId>

        <artifactId>protobuf-java</artifactId>

        <version>${version}</version>//不指名version,mvn找不到protobuf
      </dependency>

然后即可运行

mvn package -Pdist,native,docs -DskipTests -Dtar