搭建本地CDH 安装中心

搭建思路：

对于搭建本地安装中心，就要把远程的yum 源里面所有的资源 ，先下载到本地。 之后可以用Nginx，或者Apache 搭建一个web 服务，把 yum 源中相应的地址，替换成 web 服务的相对应的目录就可以了。

yum 源 核心配置文件
设置yum源的配置定义文件，该文件必须存放在/etc/yum.repos.d目录下，并且要以“.repo”结尾。本文以dvd.repo为例，编辑此配置定义文件：
[dvd]  --------------->必须写的，中括号的内容可以随便写，但一定要有中括号
name = yum server  ----------->可写可不写，内容随便，主要是个提示作用
baseurl=file:///mnt/cdrom  --------------->一定要写的，定义yum源的仓库所在
enabled=1 --------------------->数字1为启用当前yum源，0为禁用，默认为1。
gpgcheck=0  ----------------------->是否检查rpm包的数字签名，数字1为检查，0为不检查，可以不写。

首先 既然要搭建 本地安装中心，需要把所有的资源 全部要下载到本地的路径下

这是 cloudera-cdh5.repo
[cloudera-cdh5]# Packages for Cloudera's Distribution for Hadoop, Version 5, on RedHat or CentOS 6 x86_64

name=Cloudera's Distribution for Hadoop, Version 5

baseurl=https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5/

gpgkey =https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera gpgcheck = 1
https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5/ 这个网址 就是baseurl ，就是仓库的根地址。 下面有几个文件夹 repodata, RPMS , SRPMS ，只需要把这些文件夹下面所有的东西，下载下来即可，放到本地的目录下面。

#!/bin/bash
#
# @file
#   cdh5_rhel6-downloads.sh
#
# downloads all from CDH_URL_PREFIX:
#   http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/ #
################################################################################
# specify where you want to save downloaded packages here:
#
PATH_MUST_BE_EXSITED="../libs/cdh"

[ -d ../libs/cdh ]|| mkdir -p ../libs/cdh
# get real path from relative path
function real_path() {
\cd "$1"
/bin/pwd
}

# server dist resources:
#
CDH_URL_PREFIX="http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh"

CDH_GPGKEY=$CDH_URL_PREFIX"/RPM-GPG-KEY-cloudera"
CDH_REPO=$CDH_URL_PREFIX"/cloudera-cdh5.repo"

CDH5_REPODATA=$CDH_URL_PREFIX"/5/repodata/"
CDH5_RPMS_NOARCH=$CDH_URL_PREFIX"/5/RPMS/noarch/"
CDH5_RPMS_X86_64=$CDH_URL_PREFIX"/5/RPMS/x86_64/"

# source packages not used:
CDH5_SRPMS=$CDH_URL_PREFIX"/5/SRPMS/"

# get local absolute path for storing the downloaded:
CDH5_LOCALPATH=$(real_path $PATH_MUST_BE_EXSITED)
echo "**** downloaded packages will be stored in folder: "$CDH5_LOCALPATH

# first we get index pages:
#
repodata_html=$CDH5_LOCALPATH"/.repodata.index.html"
x86_64_html=$CDH5_LOCALPATH"/.x86_64.index.html"
noarch_html=$CDH5_LOCALPATH"/.noarch.index.html"
srpms_html=$CDH5_LOCALPATH"/.srpms.index.html"

wget -c $CDH5_REPODATA -P $CDH5_LOCALPATH -O $repodata_html
wget -c $CDH5_RPMS_NOARCH -P $CDH5_LOCALPATH -O $noarch_html
wget -c $CDH5_RPMS_X86_64 -P $CDH5_LOCALPATH -O $x86_64_html
wget -c $CDH5_SRPMS   -P $CDH5_LOCALPATH  -O $srpms_html

wget -c $CDH_GPGKEY -P $CDH5_LOCALPATH
wget -c $CDH_REPO -P $CDH5_LOCALPATH

# download repodata
# CDH5_REPODATA
repodata_dir=$CDH5_LOCALPATH"/5/repodata"
mkdir -p $repodata_dir
echo -e "process file: '$repodata_html'"
while read line
do
# start with: <td><a href="
a=`echo $line | sed -n '/<td><a href="/p'`

if [ -n "$a" ]; then
b=`echo $a | sed -n '/Parent Directory/p'`

# do including: Parent Directory
if [ -z "$b" ]; then
# end with: </a></td>
b=`echo $a | sed -n '/<\/a><\/td>/p'`

if [ -n "$b" ]; then
a=`echo $a | sed -e 's/.*<td><a href="//;s/">.*//'`
url=$CDH5_REPODATA$a

echo -e "download: $url"

wget -c $url -P $repodata_dir -O $repodata_dir/$a
fi
fi
fi
done < $repodata_html

# download noarch
# CDH5_RPMS_NOARCH
noarch_dir=$CDH5_LOCALPATH"/5/RPMS/noarch"
mkdir -p $noarch_dir
echo -e "process file: '$noarch_html'"
while read line
do
# start with: <td><a href="
a=`echo $line | sed -n '/<td><a href="/p'`

if [ -n "$a" ]; then
b=`echo $a | sed -n '/Parent Directory/p'`

# do including: Parent Directory
if [ -z "$b" ]; then
# end with: </a></td>
b=`echo $a | sed -n '/<\/a><\/td>/p'`

if [ -n "$b" ]; then
a=`echo $a | sed -e 's/.*<td><a href="//;s/">.*//'`
url=$CDH5_RPMS_NOARCH$a

echo -e "download: $url"

wget -c $url -P $noarch_dir -O $noarch_dir/$a
fi
fi
fi
done < $noarch_html

# download x86_64
# CDH5_RPMS_X86_64
x86_64_dir=$CDH5_LOCALPATH"/5/RPMS/x86_64"
mkdir -p $x86_64_dir
echo -e "process file: '$x86_64_html'"
while read line
do
# start with: <td><a href="
a=`echo $line | sed -n '/<td><a href="/p'`

if [ -n "$a" ]; then
b=`echo $a | sed -n '/Parent Directory/p'`

# do including: Parent Directory
if [ -z "$b" ]; then
# end with: </a></td>
b=`echo $a | sed -n '/<\/a><\/td>/p'`

if [ -n "$b" ]; then
a=`echo $a | sed -e 's/.*<td><a href="//;s/">.*//'`
url=$CDH5_RPMS_X86_64$a

echo -e "download: $url"

wget -c $url -P $x86_64_dir -O $x86_64_dir/$a
fi
fi
fi
done < $x86_64_html

#download  srpms
#SRPMS
srpms_dir=$CDH5_LOCALPATH"/5/SRPMS"
mkdir  -p $srpms_dir
echo -e "processfile: '$srpms_html'"
while read line
do
# start with: <td><a href="
a=`echo $line | sed -n '/<td><a href="/p'`

if [ -n "$a" ]; then
b=`echo $a | sed -n '/Parent Directory/p'`

# do including: Parent Directory
if [ -z "$b" ]; then
# end with: </a></td>
b=`echo $a | sed -n '/<\/a><\/td>/p'`

if [ -n "$b" ]; then
a=`echo $a | sed -e 's/.*<td><a href="//;s/">.*//'`
url=$CDH5_SRPMS$a

echo -e "download: $url"

wget -c $url -P $srpms_dir -O $srpms_dir/$a
fi
fi
fi

done < $srpms_html

# TODO: do we need to check all packages?
# remove index pages:

read -t 5  -p " do we need to check all packages? (y/n) " choice

if [ "$choice" == 'y']
then
rm -f $repodata_html $x86_64_html $noarch_html
else
echo "you are slow . all packages are not  deleted."

fi
echo "download all packages successfully."

下载 完成后 会在 当前路径下面 有 ../libs/cdn 下面 所有的 信息 都在里面了。
把 cdn 目录所有的东西， 上传到web 服务器上，就可以了。

以Apache 为例， 网站根目录： /var/www/html/ 在根目录创建一个文件夹，
mkdir cdn
把所有的东西，全部放进去。
之后 ，写一个配置文件，就可以了，如下。

cd /etc/yum.repos.d/
只需要这修改即可
vim cloudera-local-cdh5.repo
[cloudera-local-cdh5]# Packages for Cloudera's Distribution for Hadoop, Version 5, on RedHat or CentOS 6 x86_64

name=Cloudera's Distribution for Hadoop, Version 5

baseurl=http://你的网站根目录/cdh/5/

gpgkey =http://你的网站根目录/cdh/RPM-GPG-KEY-cloudera gpgcheck = 1
enabled =1

之后 ，更新一下缓存
yum clean all
yum makecache
yum list 看一下，有没有 cloudera-local-cdh5

yum repolist all

下面来测试一下：







参考文档：
cdh5 hadoop redhat 本地仓库配置
http://blog.csdn.net/ubuntu64fan/article/details/42027881

http://blog.csdn.net/jiedushi/article/details/8630686