您的位置:首页 > 运维架构 > Shell

shell脚本语言编程进阶实战

2017-10-22 17:28 801 查看
古人云“工欲善其事,必先利其器”。可以说,shell脚本语言编程就是系统管理员的一个编程利器、一个好帮手。熟练掌握和运用shell脚本语言编程,对于一个合格系统管理员来讲,可以说是基本素养。
shell脚本语言,可实现复杂程序逻辑,数据处理。除去强大的编程功能外,其无需编译环境的解释运行的特色,更显示其强大的灵活性。
学习shell编程,好处很多,但是困难是什么呢?有的说,没有shell编程学习范例。其实,在linux操作系统内,存在大量的脚本学习范例可供学习借鉴。
有的讲,没有编程需求。仔细回想一下,很多工作包含大量复杂的操作,是否有必要通过逻辑条件判断和数据抽取处理实现,并封装成自动化的脚本,以供今后重复调用。
在掌握shell编程的基础指令、正则表达和逻辑判断后,通过一些例子的练习,实现用户需求后,就可以达到shell脚本语言编程能力的进一步提升。.
本文向给出以下思路和框架。

实战.格式化文本数据
一些指令的多行输出结果,或一些文件的多行内容,可能已经结构化好了,但大部分没有格式化。所以编程进阶面对的一个重要问题,是抽取有用的数据并格式化数据,以便后续的加工处理。
格式化文本类数据基本方法如下:
(1)筛选数据
主要是运用grep、sed,抽取有用内容、剔除无用内容
# df -k|sed 1,2d 剔除指令输出中的前两行
# cat /etc/host| grep -v ^# |grep -v ^$ 剔除文件中的空行和注释行
# cat /etc/host| grep -v ^# |grep -v ^$ 剔除文件中的空行和注释行

# df -k |grep '[0-9][0-9]%' 抽取指令输出中带%且为2位数的行
# awk -F ":" '{print $1}' /etc/passwd 抽取文件的第一列

# df -k |grep '[0-9][0-9]%' |sed -e 's/%//g' 剔除指令输出中中%字符
# sed -i 's/APPLICATION//g' /etc/hosts 剔除文件中APPLICATION字符

(2)结构化数据
主要利用awk的指定字段分割符方法,完成数据的结构化
# ip a |grep "<" |awk -F ":" '{print $2}' |grep eth

(3)排序数据
主要是运用sort,对于文件差异的比对非常有用
# awk -F ":" '{print $1}' /etc/passwd |sort 按正序排列用户
# awk -F ":" '{print $1,$3}' /etc/group|sort -rn -k 2 按倒序排列用户组ID

实战.LINUX系统CPU空闲率指标检查

例:sar指令连续执行6次,求字段9%idle的最小值,如果小于80,则提示异常,否则显示正常:
脚本实现了外部参数的代入,通过统计后,得出判断结果
# cat check.sh
CPU_IDLE_STR="检查CPU空闲IDLE,大于百分"
CPU_IDLE_VAL=80
sar 1 6 |sed 1,3d| awk -v var1=$CPU_IDLE_STR -v var2=$CPU_IDLE_VAL 'BEGIN { min=0} {if ($9 < min ) min=(min<$9)?min:$9} END {if (min < var2) {printf "\t[ %-40s %-02d]\t\t*异常*\t\t当前值[ %-02d ] \n",var1,var2,min } else {printf "\t[ %-40s %-02d ]\t\t正 常\t\t当前值[ %-02d ] \n",var1,var2,min }}'

# sh check.sh

例:sar指令连续执行6次,求字段9%idle的平均值,如果小于80,则提示异常,否则显示正常:
脚本实现了外部参数的代入,通过统计后,得出判断结果
# cat check.sh
CPU_IDLE_STR="检查CPU空闲IDLE,大于百分"
CPU_IDLE_VAL=80
sar 1 6 |sed 1,3d| awk -v var1=$CPU_IDLE_STR -v var2=$CPU_IDLE_VAL 'BEGIN { sum=0;count=0 } {sum +=$9; count++;} END {if (sum/count < var2) {printf "\t[ %-40s %-02d]\t\t*异常*\t\t当前值[ %-02d ] \n",var1,var2,sum/count } else {printf "\t[ %-40s %-02d ]\t\t正 常\t\t当前值[ %-02d ] \n",var1,var2,sum/count }}'

# sh check.sh

例:统计不同日期内的报错
# cat m1.txt
Server1023/Server1023.log,Oct 17, 2017 7:31:12 PM GMT+08:00,Error> <Server1023>
Server1023/Server1023.log,Oct 17, 2017 7:32:12 PM GMT+08:00,Error> <Server1023>
Server1023/Server1023.log,Oct 17, 2017 8:05:20 PM GMT+08:00,Server1023> <[STUCK]
Server1113/Server1113.log,Sep 25, 2017 9:40:32 AM GMT+08:00,Error> <Server1113>
Server1113/Server1113.log,Sep 25, 2017 9:41:32 AM GMT+08:00,Error> <Server1113>
Server1113/Server1113.log,Sep 25, 2017 9:58:08 AM GMT+08:00,Server1113> <[STUCK]
Server1113/Server1113.log,Sep 29, 2017 3:59:22 PM GMT+08:00,Error> <Server1113>
Server1113/Server1113.log,Sep 29, 2017 4:00:22 PM GMT+08:00,Error> <Server1113>

# awk -F "," 'BEGIN { max="Oct 17" ;count=0 } { if ($2 == max ) { ++count } else { printf " %s %d \n",max,count ;max=$2;count=1} } END { printf " %s %d \n",max,count }' m1.txt
Oct 17 3
Sep 25 3
Sep 29 2

例:统计不同错误代码出现的次数
# cat m1.txt
####<Apr 7, 2017 2:07:27 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491545247275> <BEA-310003> <Free memory in the server is 4,905,048 bytes. There is danger of OutOfMemoryError>
####<Apr 8, 2017 9:06:33 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491613593096> <BEA-310003> <Free memory in the server is 4,728,536 bytes. There is danger of OutOfMemoryError>
####<Apr 9, 2017 3:46:38 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491680798625> <BEA-310003> <Free memory in the server is 4,992,928 bytes. There is danger of OutOfMemoryError>
####<Apr 9, 2017 10:54:44 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491749684634> <BEA-310003> <Free memory in the server is 4,772,736 bytes. There is danger of OutOfMemoryError>
####<Apr 10, 2017 5:51:50 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491817910249> <BEA-310003> <Free memory in the server is 4,794,072 bytes. There is danger of OutOfMemoryError>
####<Apr 11, 2017 12:51:55 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491886315817> <BEA-310003> <Free memory in the server is 4,960,776 bytes. There is danger of OutOfMemoryError>
####<Apr 12, 2017 7:52:01 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491954721248> <BEA-310003> <Free memory in the server is 4,989,464 bytes. There is danger of OutOfMemoryError>
####<Apr 13, 2017 2:55:06 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492023306984> <BEA-310003> <Free memory in the server is 4,845,776 bytes. There is danger of OutOfMemoryError>
####<Apr 13, 2017 9:59:12 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492091952865> <BEA-310003> <Free memory in the server is 4,833,352 bytes. There is danger of OutOfMemoryError>
####<Apr 14, 2017 5:00:05 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492160405970> <BEA-310003> <Free memory in the server is 4,979,432 bytes. There is danger of OutOfMemoryError>
####<Apr 15, 2017 12:04:11 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492229051734> <BEA-310003> <Free memory in the server is 4,957,360 bytes. There is danger of OutOfMemoryError>
####<Apr 16, 2017 7:08:17 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492297697464> <BEA-310003> <Free memory in the server is 4,798,184 bytes. There is danger of OutOfMemoryError>
####<Sep 23, 2017 9:41:39 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1506174099718> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>
####<Sep 23, 2017 9:41:39 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1506174099719> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>
####<May 27, 2017 9:36:29 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495892189359> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>
####<May 27, 2017 9:36:29 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495892189360> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>
####<May 27, 2017 10:02:56 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495893776640> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>
####<May 27, 2017 10:02:56 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495893776641> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>
####<May 28, 2017 7:04:24 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495969464161> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>
####<May 28, 2017 7:04:24 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495969464161> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>

# awk -F"<" '/BEA/ {print $6,$13 } END {}' o1.txt |awk '{printf "\t%s %s %s\n",S[$NF],$1,$2;++S[$NF];printf "\t %s %s %s\n",S[$NF],$1,$2 } END {for(a in S) printf "END %s %d \n",a,S[a]}'
END BEA-000415> 4
END BEA-000416> 4
END BEA-310003> 12
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签:  管理员 系统 shell