linux+shell+脚本+过滤NetScreen防火墙日志

一直想学习linux，可是没得时间。前二天，中华财险要求二天现场支持，这二天的时间，看了一些学习资料。看到公司的防火墙日志，试着过滤一下。

防火墙日志如下：

2011-09-30 00:00:20 Local0.Notice 10.2.0.254 ns50: NetScreen device_id=0019022004000299 [Root]system-notification-00257(traffic): start_time="2011-09-30 00:01:05" duration=15 policy_id=103 service=http proto=6 src
zone=Trust dst zone=Untrust action=Permit sent=2683 rcvd=766 src=10.100.1.43 dst=119.188.11.3 src_port=4048 dst_port=80 src-xlated ip=218.206.244.202 port=4679 dst-xlated ip=119.188.11.3 port=80 session_id=61727 reason=Close - AGE OUT<000>2011-09-30 00:00:20
Local0.Notice 10.2.0.254 ns50: NetScreen device_id=0019022004000299 [Root]system-notification-00257(traffic): start_time="2011-09-30 00:01:05" duration=15 policy_id=103 service=http proto=6 src zone=Trust dst zone=Untrust action=Permit sent=2674 rcvd=766
src=10.100.1.43 dst=119.188.11.3 src_port=4045 dst_port=80 src-xlated ip=218.206.244.202 port=15311 dst-xlated ip=119.188.11.3 port=80 session_id=62271 reason=Close - AGE OUT<000>2011-09-30 00:00:20 Local0.Notice 10.2.0.254 ns50: NetScreen device_id=0019022004000299
[Root]system-notification-00257(traffic): start_time="2011-09-30 00:01:05" duration=15 policy_id=103 service=http proto=6 src zone=Trust dst zone=Untrust action=Permit sent=2645 rcvd=766 src=10.100.1.43 dst=119.188.11.3 src_port=4044 dst_port=80 src-xlated
ip=218.206.244.202 port=14295 dst-xlated ip=119.188.11.3 port=80 session_id=59240 reason=Close - AGE OUT<000>2011-09-30 00:00:20 Local0.Notice 10.2.0.254 ns50: NetScreen device_id=0019022004000299 [Root]system-notification-00257(traffic): start_time="2011-09-30
00:01:05" duration=15 policy_id=103 service=http proto=6 src zone=Trust dst zone=Untrust action=Permit sent=1485 rcvd=482 src=10.100.1.43 dst=119.188.11.3 src_port=4051 dst_port=80 src-xlated ip=218.206.244.202 port=13926 dst-xlated ip=119.188.11.3 port=80
session_id=54785 reason=Close - AGE OUT<000>2011-09-30 00:00:20 Local0.Notice 10.2.0.254 ns50: NetScreen device_id=0019022004000299 [Root]system-notification-00257(traffic): start_time="2011-09-30 00:01:05" duration=15 policy_id=103 service=http
proto=6 src zone=Trust dst zone=Untrust action=Permit sent=2682 rcvd=766 src=10.100.1.43 dst=119.188.11.3 src_port=4046 dst_port=80 src-xlated ip=218.206.244.202 port=13692 dst-xlated ip=119.188.11.3 port=80 session_id=60623 reason=Close - AGE OUT<000>2011-09-30
00:00:20 Local0.Notice 10.2.0.254 ns50: NetScreen device_id=0019022004000299 [Root]system-notification-00257(traffic): start_time="2011-09-30 00:01:05" duration=15 policy_id=103 service=http proto=6 src zone=Trust dst zone=Untrust action=Permit
sent=2605 rcvd=766 src=10.100.1.43 dst=119.188.11.3 src_port=4043 dst_port=80 src-xlated ip=218.206.244.202 port=13520 dst-xlated ip=119.188.11.3 port=80 session_id=62996 reason=Close - AGE OUT<000>

想获得每条日志的sent 数据，Recv数据，src源地址及dst目的地址，脚本如下：

#!/bin/sh

if [ ! -d /var/tmp ] ; then mkdir /var/tmpfi

if [ -e /var/tmp/sysn ] ; then rm /var/tmp/sysnfi

#$1为命令行的每一个参数，这里是防火墙日志的文件路经

echo " awk { for(i=1;i<=NF;i++) { if( $i~ /sent/ ) print $i,i++,$i,i++,$i,i++,$i } } ' $1 | awk '{ print $1,$3,$5,$7 }' >/var/tmp/sysn" echo -e "..................................."

#按照模式取出字符串 类似sent=1132 recv=3434 src=10.100.1.32 dst=211.138.24.66

awk '{ for(i=1;i<=NF;i++) { if( $i~ /sent/ ) print $i,i++,$i,i++,$i,i++,$i } } ' $1 | awk '{ print $1,$3,$5,$7 }' >/var/tmp/sysn if [ -e /var/tmp/sysnn ] ; then rm /var/tmp/sysnnfi

echo " sed 's/=/ /g' /var/tmp/sysn >/var/tmp/sysnn" echo -e "..................................."

#将=换成空格

sed 's/=/ /g' /var/tmp/sysn >/var/tmp/sysnn

if [ -e /var/tmp/sysnnn ] ; then rm /var/tmp/sysnnnfi

echo "awk '{ sent[$6] += $2;Recv[$6] += $4 } END { for(i in sent) print i,"\t\t", sent[i],"\t\t",Recv[i] }' /var/tmp/sysnn >/var/tmp/sysnnn" echo -e "..................................."

#统计每个地址的sent和recv总数awk '{ sent[$6] += $2;Recv[$6] += $4 } END { for(i in sent) print i,"\t\t", sent[i],"\t\t",Recv[i] }' /var/tmp/sysnn >/var/tmp/sysnnn

if [ -e /var/tmp/sysnnnn ] ; then

rm /var/tmp/sysnnnn

fi

#按sent排序 cat /var/tmp/sysnnn | sort -n -r -k 2 | grep '^10\.' >/var/tmp/sysnnnn

/bin/echo -e "IP\t\t\tSend bytes(B)\t\tRecv bytes(B)\n====================================================================="

#命令行第二个参数，按recv排序

if [ "$2" = "recv" ] ; then cat /var/tmp/sysnnnn | sort -n -r -k 3else cat /var/tmp/sysnnnnfi

if [ -e /var/tmp/sysn ] ; then rm /var/tmp/sysnfi

if [ -e /var/tmp/sysnn ] ; then rm /var/tmp/sysnnfi

if [ -e /var/tmp/sysnnn ] ; then rm /var/tmp/sysnnnfi

if [ -e /var/tmp/sysnnnn ] ; then rm /var/tmp/sysnnnnfi

应用如下：

./syslogana /usr/Syslog2011-09-30.txt --按sent排序

或./syslogana /usr/Syslog2011-09-30.txt
recv --按recv排序

[orcle@localhost ~]$ ./syslogana /usr/Syslog2011-09-30.txt

awk { for(i=1;i<=NF;i++) { if( ~ /sent/ ) print ,i++,,i++,,i++, } } ' Syslog2011-09-30.txt | awk '{ print Syslog2011-09-30.txt,,, }' >/var/tmp/sysn...................................

sed 's/=/ /g' /var/tmp/sysn >/var/tmp/sysnn...................................

awk '{ sent[] += ;Recv[] += } END { for(i in sent) print i,tt, sent[i],tt,Recv[i] }' /var/tmp/sysnn >/var/tmp/sysnnn...................................

IP Send bytes(B) Recv bytes(B)=====================================================================10.2.0.195 389190206 3.21879e+0910.2.0.230 133985217
133386378710.2.0.240 86287521 50698167110.100.1.240 69406016 13480948610.2.0.249 56816187 14380941210.2.0.245 40095561 5869195010.2.0.228
36652824 18304863010.2.0.194 27172677 8062195710.2.0.252 23434488 9307896210.100.5.252 20701571 14683126610.2.0.241 18873421 65888402