Centos上源码安装Hadoop2.7.2

原文地址：http://happyshome.cn/blog/deploy/centos/hadoop2.7.2.html




本篇文章中介绍了在Centos7中，如何编译安装Hadoop2.7.2源码。

希望本篇文章中介绍的具体操作步骤能够对你有所帮助。

1. 安装依赖的软件包

1.1. 安装Java

Hadoop2.7依赖Java7，关于Java7版本的选择，Hadoop官方文章有测试说明，更详细的可以点击这里查看。

Version	Status	Reported By
oracle 1.7.0_15	Good	Cloudera
oracle 1.7.0_21	Good (4)	Hortonworks
oracle 1.7.0_45	Good	Pivotal
openjdk 1.7.0_09-icedtea	Good (5)	Hortonworks

这里选择的是oracle 1.7.0_45，关于Java详细的安装你可以查看 在Linux操作系统下安装JAVA8，这里我只做简要的说明。

cd /app/soft/
wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/7u45-b18/jdk-7u45-linux-x64.tar.gz" -O jdk-7u45-linux.tar.gz
tar xvf jdk-7u45-linux.tar.gz

# 设置JAVA环境变量
vi /etc/profile

# 在文件末尾，添加以下内容
export JAVA_HOME=/app/soft/jdk1.7.0_45
export JRE_HOME=$JAVA_HOME/jre
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin

# 退出保存，使设置的环境变量生效
source /etc/profile

# 查看Java环境变量，Java版本
echo $JAVA_HOME
>> /app/soft/jdk1.7.0_45

java -version
>> java version "1.7.0_45"
>> Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
>> Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)

[/code]

1.2. 安装Maven

wget http://ftp.cuhk.edu.hk/pub/packages/apache.org/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz -O apache-maven-3.3.9-bin.tar.gz
tar xvf apache-maven-3.3.9-bin.tar.gz

# 设置manve环境变量
vi /etc/profile

# 在文件末尾，添加以下内容

export PATH=$PATH:/app/soft/apache-maven-3.3.9/bin
source /etc/profile

# 查看maven版本
mvn -version
>> Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00)
>> Maven home: /app/soft/apache-maven-3.3.9
>> Java version: 1.7.0_45, vendor: Oracle Corporation
>> Java home: /app/soft/jdk1.7.0_45/jre
>> Default locale: zh_CN, platform encoding: UTF-8
>> OS name: "linux", version: "3.10.0-327.10.1.el7.x86_64", arch: "amd64", family: "unix"

[/code]

1.3. 安装CMake

# 这里我们通过利用yum安装即可
yum install cmake

1.4. 安装Protocol Buffer

wget ftp://ftp.netbsd.org/pub/pkgsrc/distfiles/protobuf-2.5.0.tar.gz -O protobuf-2.5.0.tar.gz
tar xvf protobuf-2.5.0.tar.gz
cd protobuf-2.5.0/
./configure
make
make install

[/code]

1.5. 安装findbugs

wget --no-check-certificate https://sourceforge.net/projects/findbugs/files/findbugs/1.3.9/findbugs-1.3.9.tar.gz/download -O findbugs-1.3.9.tar.gz
tar xvf findbugs-1.3.9.tar.gz

vi /etc/profile

# 在文件末尾，添加以下内容
export PATH=$PATH:/app/soft/findbugs-1.3.9/bin
source /etc/profile

[/code]

2. 安装Hadoop2.7.2

#  在编译过程中为了防止Java内存溢出，需要加入以下环境变量
vi /etc/profile
export MAVEN_OPTS="-Xmx2048m -XX:MaxPermSize=512m"
source /etc/profile

wget http://apache.01link.hk/hadoop/common/hadoop-2.7.2/hadoop-2.7.2-src.tar.gz -O hadoop-2.7.2-src.tar.gz
tar xvf hadoop-2.7.2-src.tar.gz
cd hadoop-2.7.2-src/
mvn package -Pdist,native -DskipTests -Dtar

[/code]
如果中途卡住，建议你ctrl ＋ c，然后接着重来，我折腾了一上午终于看到了以下输出



接下来我们利用编译输出后的架包查看版本信息

#  验证下Hadoop的版本
cd /app/soft/hadoop-2.7.2-src/hadoop-dist/target/hadoop-2.7.2/
bin/hadoop version
>> Hadoop 2.7.2
>> Subversion Unknown -r Unknown
>> Compiled by root on 2016-03-31T11:23Z
>> Compiled with protoc 2.5.0
>> From source with checksum d0fda26633fa762bff87ec759ebe689c
>> This command was run using /app/soft/hadoop-2.7.2-src/hadoop-dist/target/hadoop-2.7.2/share/hadoop/common/hadoop-common-2.7.2.jar

# 查看编译后的Hadoop适用于32位还是64位的系统
file lib//native/*
>> lib//native/libhadoop.a: current ar archive
>> lib//native/libhadooppipes.a: current ar archive
>> lib//native/libhadoop.so: symbolic link to 'libhadoop.so.1.0.0'
>> lib//native/libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=353e83dfbc35556442c50876509cbc4861ba0fd7, not stripped
>> lib//native/libhadooputils.a: current ar archive
>> lib//native/libhdfs.a: current ar archive
>> lib//native/libhdfs.so: symbolic link to 'libhdfs.so.0.0.0'
>> lib//native/libhdfs.so.0.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=a9a625272a853137d99b59e9c894e7db3a2446ac, not stripped