LVS构架负载均衡Linux集群系统

本人初学linux，在领导督促和工作压力下，按照前人部署手册和强大搜索引擎帮助下，用lvs和mon实现的nat负载均衡，vs是suse 10.2，realserver是windows2003的IIS部署。

1 安装
1.1 网络拓扑图



1.1 检查内核是否支持lvs选项

必须在此路径下执行命令

选择Networking,如下图所示:



选择Networking options,如下图所示



选择IP:Virtual Server Configuration



如下图所示,IPVS scheduler中,包含全部的负载均衡策略(<M>为modularizes features)



1.3 安装IP虚拟服务器软件ipvsadm
Ipvsadm在安装CD1的/suse/i586目录下.需要安装为

s193000012:~ # rpm -icvh ipvsadm-1.24-123.2.i586.rpm

安装完，执行ipvsadm命令，应该有类似如下的信息出现：

s193000012:~ # ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn

出现类似以上信息，表明支持LVS的内核和配置工具ipvsadm已完全安装，这台Director机器已经初步安装完成，已具备构架各种方式的集群的条件。
1.4 安装mon
1.4.1 检查mon是否安装

s193000012:~ # rpm -qa |grep mon perl-Net-Daemon-0.38-61.2 mono-data-1.2.2-12.20 smartmontools-5.37.0.20070914-5.3 mono-core-1.2.2-12.20 mono-winforms-1.2.2-12.20 dbus-1-mono-0.60-33.22 mono-web-1.2.2-12.20 openhpi-daemon-2.10.2-0.4 mon-0.99.2-368.2

如果没有出现以上标记中内容，按照顺序安装
perl-Time-Period-1.20-317.2.i586.rpm
perl-Convert-BER-1.3101-206.2.i586.rpm
perl-Mon-0.11-310.2.i586.rpm
fping-2.4b2-15.2.i586.rpm
mon-0.99.2-368.2.i586.rpm

1.4.2 Lvs.alert脚本
/usr/lib/mon/alert.d/lvs.alert

#!/usr/bin/perl # # lvs.alert - Linux Virtual Server alert for mon # # It can be activated by mon to remove a real server when the # service is down, or add the server when the service is up. # # use Getopt::Std; getopts ("s:g:h:t:l:P:V:R:W:F:u"); #P:protocal #V:virtual_server #R:Real_server #F:forwarding $ipvsadm = "/sbin/ipvsadm"; $protocol = $opt_P; $virtual_service = $opt_V; $remote = $opt_R; if ($opt_u) { $weight = $opt_W; if ($opt_F eq "nat") { $forwarding = "-m"; } elsif ($opt_F eq "tun") { $forwarding = "-i"; } else { $forwarding = "-g"; } if ($protocol eq "tcp") { system("$ipvsadm -a -t $virtual_service -r $remote $forwarding -w $weight"); } else { system("$ipvsadm -a -u $virtual_service -r $remote $forwarding -w $weight"); } } else { if ($protocol eq "tcp") { system("$ipvsadm -d -t $virtual_service -r $remote"); } else { system("$ipvsadm -d -u $virtual_service -r $remote"); } };

当realserver发生变化，通过以上警告，删除或添加ipvsadm的记录

2 Mon配置文件
/etc/mon/mon.cf

# # global options # cfbasedir = /etc/mon alertdir = /usr/lib/mon/alert.d mondir = /usr/lib/mon/mon.d statedir = /var/lib/mon logdir = /var/log maxprocs = 20 histlength = 100 historicfile = mon_history.log randstart = 60s authtype = getpwnam dtlogging = yes # service telnet #pool_ip1 hostgroup pool_ip1 10.194.3.41 watch pool_ip1 service http interval 1s monitor http.monitor period wd {Mon-Sun} alert lvs.alert -P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat upalert lvs.alert -P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat -u 1 #pool_ip2 hostgroup pool_ip2 10.194.3.42 watch pool_ip2 service http interval 1s monitor http.monitor period wd {Mon-Sun} alert lvs.alert -P tcp -V 10.193.0.13:80 -R 10.194.3.42:80 -W 10 -F nat upalert lvs.alert -P tcp -V 10.193.0.13:80 -R 10.194.3.42:80 -W 10 -F nat -u 1

以上是realserver发生变化，产生log，并调用lvs.alert

upalert lvs.alert -P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat -u 1
发生down 协议 vs realserver 权重 负载方式 恢复标志

2.1 启动mon服务

#service mon start

禁用服务
Service mon stop

重启服务
Service mon restart

2.1.1 日志

/var/log/ mon_history.log

alert pool_ip2 http 1255580913 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.42:80 -W 10 -F nat) 10.194.3.42 upalert pool_ip2 http 1255580915 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.42:80 -W 10 -F nat -u 1) 10.194.3.42

从上面日志里可以看出，1条日志10.194.3.42发出报警，2条日志10.194.3.42恢复正常

2.2 设置linux为路由转发模式

编译:/etc/rc.d/rc
增加: echo "1" > /proc/sys/net/ipv4/ip_forward
保存后重启，或者执行如上命令，才能生效

test -e /etc/issue-SuSE-first-run && { cat /etc/issue-SuSE-first-run > $REDIRECT rm -f /etc/issue-SuSE-first-run 2>/dev/null } echo "1" > /proc/sys/net/ipv4/ip_forward exit 0

2.3 建立LVS可执行文件

vs_ip=10.193.0.13 vs_port=80 pool_ip1=10.194.3.41 pool_ip1_wight=10 pool_ip2=10.194.3.42 pool_ip2_wight=10 pool_port=80 #Clear all the ipvsadmin configurations ipvsadm -C # Choose the Weighted Round Robing ifconfig eth0:1 down ifconfig eth0:1 $vs_ip netmask 255.255.255.255 up ipvsadm -A -t $vs_ip:$vs_port -s wrr #ipvsadm -A -t $vs_ip:$vs_port -s lc # Set Real Server ipvsadm -a -t $vs_ip:$vs_port -r $pool_ip1:$pool_port -m -w $pool_ip1_wight ipvsadm -a -t $vs_ip:$vs_port -r $pool_ip2:$pool_port -m -w $pool_ip2_wight #Display the current configurations ipvsadm ~

以上是建立lvs可执行文件，可以放在任意目录，我保存在/etc/init.d下
需要执行lvs
当前目录>./lvs

3 测试

启动：
Service mon restart（前面已经执行过）
echo "1" > /proc/sys/net/ipv4/ip_forward（手工执行，或者重启服务器）
当前目录>./lvs（前面已经执行过）

s193000012:/etc/init.d # ipvsadm //显示内容 IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.193.0.13:http wrr -> 10.194.3.42:http Masq 10 0 1 -> 10.194.3.41:http Masq 10 1 0 s193000012:/etc/init.d # ipvsadm –ln //显示端口 IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.193.0.13:80 wrr -> 10.194.3.42:80 Masq 10 0 1 -> 10.194.3.41:80 Masq 10 1 0 s193000012:/etc/init.d # ipvsadm –lnc //显示当前连接数 IPVS connection entries pro expire state source virtual destination TCP 00:46 TIME_WAIT 10.0.100.222:1938 10.193.0.13:80 10.194.3.42:80 TCP 01:51 TIME_WAIT 10.0.100.222:1963 10.193.0.13:80 10.194.3.41:80

如果现在断开10.193.3.41
在/var/log/ mon_history.log

alert pool_ip1 http 1255580551 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat) 10.194.3.41 alert pool_ip1 http 1255580553 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat) 10.194.3.41

执行ipvsadm，发现少了10.193.3.41，这样就不会发生访问异常

s193000012:/etc/init.d # ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.193.0.13:http wrr -> 10.194.3.42:http Masq 10 0 0

恢复10.193.3.41的连接，发现upalert日志，说明已经恢复

alert pool_ip1 http 1255596062 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat) 10.194.3.41 upalert pool_ip1 http 1255596064 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat -u 1) 10.194.3.41

执行ipvsadm，10.193.3.41记录已经到ipvsadm中了

s193000012:/etc/init.d # ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.193.0.13:http wrr -> 10.194.3.41:http Masq 10 0 0 -> 10.194.3.42:http Masq 10 0 0 s193000012:/etc/init.d #

4 问题
1、realerver断开后，mon_ history.log，系统不断有警报日志弹出，执行ipvsadm，发现报警的realerver还在ipvsadm中，linux弹出

/usr/lib/mon/alert.d/lvs.alert: line 15: use: command not found /usr/lib/mon/alert.d/lvs.alert: line 16: syntax error near unexpected token `"s:g:h:t:l:P:V:R:W:F:u"' /usr/lib/mon/alert.d/lvs.alert: line 16: `getopts ("s:g:h:t:l:P:V:R:W:F:u");'

问题出在usr/lib/mon/alert.d/lvs.alert的15、16行，上网查了半天，发现可能是编码问题，我是从剪切板粘贴过来的，从alert.template复制一个lvs.alert，将use Getopt::Std;
getopts ("s:g:h:t:l:P:V:R:W:F:u");按照所需进行修改，其他内容进行粘贴，问题解决！