亚马逊AWS学习——EC2实例无法正确加载EBS卷问题的解决

原文地址：http://blog.csdn.net/dongdong9223/article/details/48051881

1 月桂女神的雀斑

谈到男女思维的差异我们常找“男人来自火星女人来自金星”的借口，其实人类本身就是远交近攻的动物。好比站在地球看月亮美得只有诗情画意，可如果是近月轨道的传真照片就现实得能让人冷噤了。跟EC2实例做些亲密接触，多少也能偷窥到点月桂女神的雀斑。

2 问题描述

正常建立EC2实例的流程很简单。其中步骤4需要为实例添加存储：



对于某些AMI（Amazon 系统映像）类型，创建好之后你会发现，本来为根目录指定了大小，还添加了新的存储卷，但是实例里面都没有正常显示。根目录一般只显示为5.7G大小。今天来说说如何让根目录显示正常（对于新建的存储卷，默认确实就是显示不出的，需要手动分区和挂载，这个以后再说）。

先说说正常创建的显示。比如创建EC2实例过程中，AMI选择默认的”快速启动”中的“Red Hat Enterprise Linux 7.1 (HVM)，SSD 卷类型 - ami-9833a1a1”：



并且将根目录设定为50G：



实例创建好了之后登陆系统，你会看到正常的存储已经设置好：



但如果AMI类型选择的是“社区AMI”中的“RHEL-6.4_HVM_GA-20141002-x86_64-11-Hourly2-GP2 - ami-7846d441”：



同样设置根目录为50G。实例创建好之后，存储显示不正常：



能够看到整个硬盘确实有50G，但只分给了根目录“/”5.7G，也就是说根目录并未得到我们分配的完整容量，磁盘空间设置有问题。

3 问题原因

究竟是什么原因导致这种情况的发生呢？这其实是由aws云的特性决定的。所谓眼见为实，不过现代社会中这句话要大打折扣，很多时候看到的结果都不是想当然如此的。EC2作为AWS的虚拟产品，其所承载操作系统的构造和使用有许多不同于传统操作系统的地方。在早些时候，AWS的EC2实例创建之后都是需要手工扩展根卷的，只不过后来发展了根卷扩展脚本可以在EC2 launch的时候自动扩展，以帮助客户减轻负担。

明白了吧，也就是说你看到的“5.7G”其实是正常的结果，只不过还需要手工做些工作，完整的“50G”反而是有脚本在背后默默奉献才做到的，正是：

爱我的人她为我付出一切，我却为我爱的人流泪狂乱心碎…

不好意思小鱼很爱唱歌的。

4 解决办法

现在明白了问题产生的原因了。有没有解决办法？当然有。其实就是说，整个根分区所在磁盘确实是有50G的，只是系统默认只给了根分区5.7G，我们需要做的就是修改根分区在分区表里起始和终止扇区的地址，使得根分区可以占据整个磁盘。AWS有手动扩展Linux分区的文档，可以参考。

有没有更简单一点的方式？答案还是有，那就是使用growpart工具（具体内容见本文最后的附录，MarkDown无法添加附件，只能全贴上来了。大家只需将其内容粘贴至一个文件，并将其命名为growpart即可）。growpart是AWS的扩展分区工具，只需简单几个命令就可以实现分区扩展，接下来讲一下使用方法。

4.1 准备growpart

登陆你的EC2实例，并将growpart脚本上传。

4.2 执行growpart

执行sh growpart -h，能够看到growpart的使用方法：



能够看到growpart命令后面“/dev/sda”是设备文件名，“1”是分区号，注意EC2下操作要使用系统权限，即加上“sudo”：



然后执行partprobe命令让内核重新读取分区表信息：

4.3 重启系统

不过此时大多数情况下lsblk命令还是无法获取最新的磁盘信息，重启一下系统：

4.4 查看扩展后分区信息

重启后重新登录，再次查看磁盘分区信息，能够看到扩展后根目录的分区覆盖了整个磁盘，为50G，达到了我们预期的效果：

5 结束语

虽然问题结束了，但是小鱼还想啰嗦几句。从使用上来看，能看得出growpart这个工具还是很方便的，但是究竟哪些系统集成了growpart，哪些系统没有集成而需要我们手动使用growpart扩展分区，这个AWS并没有给出明确的列表，大概比较新的镜像都对growpart做了集成，如RedHead 7.1，还有AWS自己的Amazon Linux什么的。在这些系统下，能够看到growpart工具的脚本位置和所属安装包，如：

RedHead 7.1下这个脚本路径和所属安装包：

# which growpart
/bin/growpart

# rpm -qf /bin/growpart
cloud-utils-growpart-0.27-13.el7.noarch

1
2
3
4
5
[/code]

Amazon Linux下这个脚本路径和所属安装包：

$ which growpart
/usr/bin/growpart

$ rpm -qf /usr/bin/growpart
cloud-disk-utils-0.27-1.4.amzn1.noarch

1
2
3
4
5
[/code]

Ubuntu Linux下这个脚本路径和所属安装包：

# which growpart
/usr/bin/growpart

# dpkg -S /usr/bin/growpart
cloud-guest-utils: /usr/bin/growpart

1
2
3
4
5
[/code]

在解决问题过程中得到了AWS的Feng L、Shun W、Bruce Li（名头好响）的不少帮助，在此表示感谢。不过，作为扩展分区这么重要的一个工具，growpart在AWS的文档中并没有被说明，脚本本身在AWS上也没有链接，新手遇到问题只有盲人摸象了。这或许是EC2值得改进的一点吧。

6 附录（growpart脚本）

#!/bin/sh
#    Copyright (C) 2011 Canonical Ltd.
#    Copyright (C) 2013 Hewlett-Packard Development Company, L.P.
#
#    Authors: Scott Moser <smoser@canonical.com>
#             Juerg Haefliger <juerg.haefliger@hp.com>
#
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, version 3 of the License.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see <http://www.gnu.org/licenses/>.

# the fudge factor. if its within this many 512 byte sectors, dont bother
FUDGE=${GROWPART_FUDGE:-$((20*1024))}
TEMP_D=""
RESTORE_FUNC=""
RESTORE_HUMAN=""
VERBOSITY=0
DISK=""
PART=""
PT_UPDATE=false
DRY_RUN=0

MBR_CHS=""
MBR_BACKUP=""
GPT_BACKUP=""
_capture=""

error() {
echo "$@" 1>&2
}

fail() {
[ $# -eq 0 ] || echo "FAILED:" "$@"
exit 2
}

nochange() {
echo "NOCHANGE:" "$@"
exit 1
}

changed() {
echo "CHANGED:" "$@"
exit 0
}

change() {
echo "CHANGE:" "$@"
exit 0
}

cleanup() {
if [ -n "${RESTORE_FUNC}" ]; then
error "***** WARNING: Resize failed, attempting to revert ******"
if ${RESTORE_FUNC} ; then
error "***** Appears to have gone OK ****"
else
error "***** FAILED! or original partition table" \
"looked like: ****"
cat "${RESTORE_HUMAN}" 1>&2
fi
fi
[ -z "${TEMP_D}" -o ! -d "${TEMP_D}" ] || rm -Rf "${TEMP_D}"
}

debug() {
local level=${1}
shift
[ "${level}" -gt "${VERBOSITY}" ] && return
if [ "${DEBUG_LOG}" ]; then
echo "$@" >>"${DEBUG_LOG}"
else
error "$@"
fi
}

debugcat() {
local level="$1"
shift;
[ "${level}" -gt "$VERBOSITY" ] && return
if [ "${DEBUG_LOG}" ]; then
cat "$@" >>"${DEBUG_LOG}"
else
cat "$@" 1>&2
fi
}

mktemp_d() {
# just a mktemp -d that doens't need mktemp if its not there.
_RET=$(mktemp -d "${TMPDIR:-/tmp}/${0##*/}.XXXXXX" 2>/dev/null) &&
return
_RET=$(umask 077 && t="${TMPDIR:-/tmp}/${0##*/}.$$" &&
mkdir "${t}" &&	echo "${t}")
return
}

Usage() {
cat <<EOF
${0##*/} disk partition
rewrite partition table so that partition takes up all the space it can
options:
-h | --help       print Usage and exit
--fudge F    if part could be resized, but change would be
less than 'F', do not resize (default: ${FUDGE})
-N | --dry-run    only report what would be done, show new 'sfdisk -d'
-v | --verbose    increase verbosity / debug
-u | --update  R  update the the kernel partition table info after growing
this requires kernel support and 'partx --update'
R is one of:
- 'auto'  : [default] update partition if possible
- 'force' : try despite sanity checks (fail on failure)
- 'off'   : do not attempt
- 'on'    : fail if sanity checks indicate no support

Example:
- ${0##*/} /dev/sda 1
Resize partition 1 on /dev/sda
EOF
}

bad_Usage() {
Usage 1>&2
error "$@"
exit 2
}

mbr_restore() {
sfdisk --no-reread "${DISK}" ${MBR_CHS} -I "${MBR_BACKUP}"
}

sfdisk_worked_but_blkrrpart_failed() {
local ret="$1" output="$2"
# exit code found was just 1, but dont insist on that
#[ $ret -eq 1 ] || return 1
# Successfully wrote the new partition table
grep -qi "Success.* wrote.* new.* partition" "$output" &&
grep -qi "BLKRRPART: Device or resource busy" "$output"
return
}

mbr_resize() {
RESTORE_HUMAN="${TEMP_D}/recovery"
MBR_BACKUP="${TEMP_D}/orig.save"

local change_out=${TEMP_D}/change.out
local dump_out=${TEMP_D}/dump.out
local new_out=${TEMP_D}/new.out
local dump_mod=${TEMP_D}/dump.mod
local tmp="${TEMP_D}/tmp.out"
local err="${TEMP_D}/err.out"

local _devc cyl _w1 heads _w2 sectors _w3 tot dpart
local pt_start pt_size pt_end max_end new_size change_info

# --show-pt-geometry outputs something like
#     /dev/sda: 164352 cylinders, 4 heads, 32 sectors/track
rqe sfd_geom sfdisk "${DISK}" --show-pt-geometry >"${tmp}" &&
read _devc cyl _w1 heads _w2 sectors _w3 <"${tmp}" &&
MBR_CHS="-C ${cyl} -H ${heads} -S ${sectors}" ||
fail "failed to get CHS from ${DISK}"

tot=$((${cyl}*${heads}*${sectors}))

debug 1 "geometry is ${MBR_CHS}. total size=${tot}"
rqe sfd_dump sfdisk ${MBR_CHS} --unit=S --dump "${DISK}" \
>"${dump_out}" ||
fail "failed to dump sfdisk info for ${DISK}"

{
echo "## sfdisk ${MBR_CHS} --unit=S --dump ${DISK}"
cat "${dump_out}"
}  >"${RESTORE_HUMAN}"
[ $? -eq 0 ] || fail "failed to save sfdisk -d output"

debugcat 1 "${RESTORE_HUMAN}"

sed -e 's/,//g; s/start=/start /; s/size=/size /' "${dump_out}" \
>"${dump_mod}" ||
fail "sed failed on dump output"

dpart="${DISK}${PART}" # disk and partition number
if [ -b "${DISK}p${PART}" -a "${DISK%[0-9]}" != "${DISK}" ]; then
# for block devices that end in a number (/dev/nbd0)
# the partition is "<name>p<partition_number>" (/dev/nbd0p1)
dpart="${DISK}p${PART}"
elif [ "${DISK#/dev/loop[0-9]}" != "${DISK}" ]; then
# for /dev/loop devices, sfdisk output will be <name>p<number>
# format also, even though there is not a device there.
dpart="${DISK}p${PART}"
fi

pt_start=$(awk '$1 == pt { print $4 }' "pt=${dpart}" <"${dump_mod}") &&
pt_size=$(awk '$1 == pt { print $6 }' "pt=${dpart}" <"${dump_mod}") &&
[ -n "${pt_start}" -a -n "${pt_size}" ] &&
pt_end=$((${pt_size}+${pt_start})) ||
fail "failed to get start and end for ${dpart} in ${DISK}"

# find the minimal starting location that is >= pt_end
max_end=$(awk '$3 == "start" { if($4 >= pt_end && $4 < min)
{ min = $4 } } END { printf("%s\n",min); }' \
min=${tot} pt_end=${pt_end} "${dump_mod}") &&
[ -n "${max_end}" ] ||
fail "failed to get max_end for partition ${PART}"

debug 1 "max_end=${max_end} tot=${tot} pt_end=${pt_end}" \
"pt_start=${pt_start} pt_size=${pt_size}"
[ $((${pt_end})) -eq ${max_end} ] &&
nochange "partition ${PART} is size ${pt_size}. it cannot be grown"
[ $((${pt_end}+${FUDGE})) -gt ${max_end} ] &&
nochange "partition ${PART} could only be grown by" \
"$((${max_end}-${pt_end})) [fudge=${FUDGE}]"

# now, change the size for this partition in ${dump_out} to be the
# new size
new_size=$((${max_end}-${pt_start}))
sed "\|^\s*${dpart} |s/${pt_size},/${new_size},/" "${dump_out}" \
>"${new_out}" ||
fail "failed to change size in output"

change_info="partition=${PART} start=${pt_start} old: size=${pt_size} end=${pt_end} new: size=${new_size},end=${max_end}"
if [ ${DRY_RUN} -ne 0 ]; then
echo "CHANGE: ${change_info}"
{
echo "# === old sfdisk -d ==="
cat "${dump_out}"
echo "# === new sfdisk -d ==="
cat "${new_out}"
} 1>&2
exit 0
fi

LANG=C sfdisk --no-reread "${DISK}" ${MBR_CHS} --force \
-O "${MBR_BACKUP}" <"${new_out}" >"${change_out}" 2>&1
ret=$?
[ $ret -eq 0 ] || RESTORE_FUNC="mbr_restore"

if [ $ret -eq 0 ]; then
:
elif $PT_UPDATE &&
sfdisk_worked_but_blkrrpart_failed "$ret" "${change_out}"; then
# if the command failed, but it looks like only because
# the device was busy and we have pt_update, then go on
debug 1 "sfdisk failed, but likely only because of blkrrpart"
else
error "attempt to resize ${DISK} failed. sfdisk output below:"
sed 's,^,| ,' "${change_out}" 1>&2
fail "failed to resize"
fi

rq pt_update pt_update "$DISK" "$PART" ||
fail "pt_resize failed"

RESTORE_FUNC=""

changed "${change_info}"

# dump_out looks something like:
## partition table of /tmp/out.img
#unit: sectors
#
#/tmp/out.img1 : start=        1, size=    48194, Id=83
#/tmp/out.img2 : start=    48195, size=   963900, Id=83
#/tmp/out.img3 : start=  1012095, size=   305235, Id=82
#/tmp/out.img4 : start=  1317330, size=   771120, Id= 5
#/tmp/out.img5 : start=  1317331, size=   642599, Id=83
#/tmp/out.img6 : start=  1959931, size=    48194, Id=83
#/tmp/out.img7 : start=  2008126, size=    80324, Id=83
}

gpt_restore() {
sgdisk -l "${GPT_BACKUP}" "${DISK}"
}

gpt_resize() {
GPT_BACKUP="${TEMP_D}/pt.backup"

local pt_info="${TEMP_D}/pt.info"
local pt_pretend="${TEMP_D}/pt.pretend"
local pt_data="${TEMP_D}/pt.data"
local out="${TEMP_D}/out"

local dev="disk=${DISK} partition=${PART}"

local pt_start pt_end pt_size last pt_max code guid name new_size
local old new change_info

# Dump the original partition information and details to disk. This is
# used in case something goes wrong and human interaction is required
# to revert any changes.
rqe sgd_info sgdisk "--info=${PART}" --print "${DISK}" >"${pt_info}" ||
RESTORE_HUMAN="${pt_info}"

debug 1 "$dev: original sgdisk info:"
debugcat 1 "${pt_info}"

# Pretend to move the backup GPT header to the end of the disk and dump
# the resulting partition information. We use this info to determine if
# we have to resize the partition.
rqe sgd_pretend sgdisk --pretend --move-second-header \
--print "${DISK}" >"${pt_pretend}" ||
fail "${dev}: failed to dump pretend sgdisk info"

debug 1 "$dev: pretend sgdisk info"
debugcat 1 "${pt_pretend}"

# Extract the partition data from the pretend dump
awk 'found { print } ; $1 == "Number" { found = 1 }' \
"${pt_pretend}" >"${pt_data}" ||
fail "${dev}: failed to parse pretend sgdisk info"

# Get the start and end sectors of the partition to be grown
pt_start=$(awk '$1 == '"${PART}"' { print $2 }' "${pt_data}") &&
[ -n "${pt_start}" ] ||
fail "${dev}: failed to get start sector"
pt_end=$(awk '$1 == '"${PART}"' { print $3 }' "${pt_data}") &&
[ -n "${pt_end}" ] ||
fail "${dev}: failed to get end sector"
pt_size="$((${pt_end} - ${pt_start}))"

# Get the last usable sector
last=$(awk '/last usable sector is/ { print $NF }' \
"${pt_pretend}") && [ -n "${last}" ] ||
fail "${dev}: failed to get last usable sector"

# Find the minimal start sector that is >= pt_end
pt_max=$(awk '{ if ($2 >= pt_end && $2 < min) { min = $2 } } END \
{ print min }' min="${last}" pt_end="${pt_end}" \
"${pt_data}") && [ -n "${pt_max}" ] ||
fail "${dev}: failed to find max end sector"

debug 1 "${dev}: pt_start=${pt_start} pt_end=${pt_end}" \
"pt_size=${pt_size} pt_max=${pt_max} last=${last}"

# Check if the partition can be grown
[ "${pt_end}" -eq "${pt_max}" ] &&
nochange "${dev}: size=${pt_size}, it cannot be grown"
[ "$((${pt_end} + ${FUDGE}))" -gt "${pt_max}" ] &&
nochange "${dev}: could only be grown by" \
"$((${pt_max} - ${pt_end})) [fudge=${FUDGE}]"

# The partition can be grown if we made it here. Get some more info
# about it so we can do it properly.
# FIXME: Do we care about the attribute flags?
code=$(awk '/^Partition GUID code:/ { print $4 }' "${pt_info}")
guid=$(awk '/^Partition unique GUID:/ { print $4 }' "${pt_info}")
name=$(awk '/^Partition name:/ { gsub(/'"'"'/, "") ; \
if (NF >= 3) print substr($0, index($0, $3)) }' "${pt_info}")
[ -n "${code}" -a -n "${guid}" ] ||
fail "${dev}: failed to parse sgdisk details"

debug 1 "${dev}: code=${code} guid=${guid} name='${name}'"

# Calculate the new size of the partition
new_size=$((${pt_max} - ${pt_start}))
old="old: size=${pt_size},end=${pt_end}"
new="new: size=${new_size},end=${pt_max}"
change_info="${dev}: start=${pt_start} ${old} ${new}"

# Dry run
[ "${DRY_RUN}" -ne 0 ] && change "${change_info}"

# Backup the current partition table, we're about to modify it
rq sgd_backup sgdisk "--backup=${GPT_BACKUP}" "${DISK}" ||
fail "${dev}: failed to backup the partition table"

# Modify the partition table. We do it all in one go (the order is
# important!):
#  - move the GPT backup header to the end of the disk
#  - delete the partition
#  - recreate the partition with the new size
#  - set the partition code
#  - set the partition GUID
#  - set the partition name
rq sgdisk_mod sgdisk --move-second-header "--delete=${PART}" \
"--new=${PART}:${pt_start}:${pt_max}" \
"--typecode=${PART}:${code}" \
"--partition-guid=${PART}:${guid}" \
"--change-name=${PART}:${name}" "${DISK}" &&
rq pt_update pt_update "$DISK" "$PART" || {
RESTORE_FUNC=gpt_restore
fail "${dev}: failed to repartition"
}

changed "${change_info}"
}

kver_to_num() {
local kver="$1" maj="" min="" mic="0"
kver=${kver%%-*}
maj=${kver%%.*}
min=${kver#${maj}.}
min=${min%%.*}
mic=${kver#${maj}.${min}.}
[ "$kver" = "$mic" ] && mic=0
_RET=$(($maj*1000*1000+$min*1000+$mic))
}

kver_cmp() {
local op="$2" n1="" n2=""
kver_to_num "$1"
n1="$_RET"
kver_to_num "$3"
n2="$_RET"
[ $n1 $op $n2 ]
}

rq() {
# runquieterror(label, command)
# gobble stderr of a command unless it errors
local label="$1" ret="" efile=""
efile="$TEMP_D/$label.err"
shift;
debug 2 "running[$label][$_capture]" "$@"
if [ "${_capture}" = "erronly" ]; then
"$@" 2>"$TEMP_D/$label.err"
ret=$?
else
"$@" >"$TEMP_D/$label.err" 2>&1
ret=$?
fi
if [ $ret -ne 0 ]; then
error "failed [$label:$ret]" "$@"
cat "$efile" 1>&2
fi
return $ret
}

rqe() {
local _capture="erronly"
rq "$@"
}

verify_ptupdate() {
local input="$1" found="" reason="" kver=""

# we can always satisfy 'off'
if [ "$input" = "off" ]; then
_RET="false";
return 0;
fi

if command -v partx >/dev/null 2>&1; then
partx --help 2>/dev/null | grep -q -- --update || {
reason="partx has no '--update' flag in usage."
found="off"
}
else
reason="no 'partx' command"
found="off"
fi

if [ -z "$found" ]; then
if [ "$(uname)" != "Linux" ]; then
reason="Kernel is not Linux per uname."
found="off"
fi
fi

if [ -z "$found" ]; then
kver=$(uname -r) || debug 1 "uname -r failed!"

if ! kver_cmp "${kver-0.0.0}" -ge 3.8.0; then
reason="Kernel '$kver' < 3.8.0."
found="off"
fi
fi

if [ -z "$found" ]; then
_RET="true"
return 0
fi

case "$input" in
on) error "$reason"; return 1;;
auto)
_RET="false";
debug 1 "partition update disabled: $reason"
return 0;;
force)
_RET="true"
error "WARNING: ptupdate forced on even though: $reason"
return 0;;
esac
error "unknown input '$input'";
return 1;
}

pt_update() {
local dev="$1" part="$2" update="${3:-$PT_UPDATE}"
if ! $update; then
return 0
fi
partx --update "$part" "$dev"
}

has_cmd() {
command -v "${1}" >/dev/null 2>&1
}

pt_update="auto"
while [ $# -ne 0 ]; do
cur=${1}
next=${2}
case "$cur" in
-h|--help)
Usage
exit 0
;;
--fudge)
FUDGE=${next}
shift
;;
-N|--dry-run)
DRY_RUN=1
;;
-u|--update|--update=*)
if [ "${cur#--update=}" != "$cur" ]; then
next="${cur#--update=}"
else
shift
fi
case "$next" in
off|auto|force|on) pt_update=$next;;
*) fail "unknown --update option: $next";;
esac
;;
-v|--verbose)
VERBOSITY=$(($VERBOSITY+1))
;;
--)
shift
break
;;
-*)
fail "unknown option ${cur}"
;;
*)
if [ -z "${DISK}" ]; then
DISK=${cur}
else
[ -z "${PART}" ] || fail "confused by arg ${cur}"
PART=${cur}
fi
;;
esac
shift
done

[ -n "${DISK}" ] || bad_Usage "must supply disk and partition-number"
[ -n "${PART}" ] || bad_Usage "must supply partition-number"

has_cmd "sfdisk" || fail "sfdisk not found"

[ -e "${DISK}" ] || fail "${DISK}: does not exist"

[ "${PART#*[!0-9]}" = "${PART}" ] || fail "partition-number must be a number"

verify_ptupdate "$pt_update" || fail
PT_UPDATE=$_RET

debug 1 "update-partition set to $PT_UPDATE"

mktemp_d && TEMP_D="${_RET}" || fail "failed to make temp dir"
trap cleanup EXIT

# get the ID of the first partition to determine if it's MBR or GPT
id=$(sfdisk --id --force "${DISK}" 1 2>/dev/null) ||
fail "unable to determine partition type"

if [ "${id}" = "ee" ] ; then
has_cmd "sgdisk" || fail "GPT partition found but no sgdisk"
debug 1 "found GPT partition table (id = ${id})"
gpt_resize
else
debug 1 "found MBR partition table (id = ${id})"
mbr_resize
fi

# vi: ts=4 noexpandtab