在Linux下使用LVS+keepalived搭建高可用集群 推荐

在Linux下使用LVS+keepalived搭建高可用集群

一． LVS+Keepalived介绍

LVS
LVS是Linux Virtual Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立，是中国国内最早出现的自由软件项目之一。目前有三种IP负载均衡技术（VS/NAT、VS/TUN和VS/DR）；

十种调度算法（rrr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq）。
LVS中文支持站点：http://zh.linuxvirtualserver.org/
Keepalived
Keepalived在这里主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现.
Keepalived官方站点：http://www.keepalived.org/

二． 实验所用软件环境及拓扑图

实验软件环境：
1. 系统Centos 5.6 x64
2. 软件LVS,Keepalived，Apache
3. 拓扑图如下：




4. IP配置信息
VIP(虚拟IP)： 192.168.1.35
DIP1(主调度器)：192.168.1.31
DIP2(备份调度)：192.1681.32
RIP1(主WEB) ：192.168.1.33
RIP2(副WEB) ：192.168.1.34
网 关 ：192.168.1.1

三．开始安装LVS+Keepalived

1.下载相关软件包

mkdir /soft //建立一个存放软件的目录
cd /soft
wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz

2.安装LVS和Keepalived

lsmod | grep ip_vs //查看内核中是否已经加载ip_vs模块
uname -r //查看当前内核版本号
2.6.18-238.el5 //当前内核版本号
ln -s /usr/src/kernels/2.6.18-238.el5-x86_64/ /usr/src/linux/ //将内核模块做软连接至/usr/src/目录下 并改名为/usr/src/linux/这样做的好处是，在编译keepalived的时候可以直接找到内核文件，而不用指定内核路径，如果不做此项操作，则在编译时需指定内核路径
tar xvf ipvsadm-1.24.tar.gz
cd ipvsadm-1.24
make && make install
find / -name ipvsadm //查看ipvsadm的位置
tar xvf keepalived-1.1.15.tar.gz
cd keepalived-1.1.15
./configure
make && make install
find / -name keepalived //查看keepalived位置
cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/

cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/

mkdir /etc/keepalived

cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/

cp /usr/local/sbin/keepalived /usr/sbin/

service keepalived start|stop //做成系统启动服务方便管理.

3. 配置LVS实现负载均衡

a）LVS-DR，配置LVS脚本实现负载均衡 ,此脚本只为测试LVS是否成功，后面主备切换使用keepalived，此脚本不用。

vi /usr/local/sbin/lvs-dr.sh #!/bin/bash # description: start LVS of DirectorServer #Written by :Supernic #www.supernic.cn GW=192.168.2.253 # website director vip. WEB_VIP=192.168.1.35 WEB_RIP1=192.168.1.33 WEB_RIP2=192.168.1.34 . /etc/rc.d/init.d/functions logger $0 called with $1 case "$1" in start) # Clear all iptables rules. /sbin/iptables -F # Reset iptables counters. /sbin/iptables -Z # Clear all ipvsadm rules/services. /sbin/ipvsadm -C #set lvs vip for dr /sbin/ipvsadm --set 30 5 60 /sbin/ifconfig eth0:0 $WEB_VIP broadcast $WEB_VIP netmask 255.255.255.255 up /sbin/route add -host $WEB_VIP dev eth0:0 /sbin/ipvsadm -A -t $WEB_VIP:80 -s wrr -p 3 /sbin/ipvsadm -a -t $WEB_VIP:80 -r $WEB_RIP1:80 -g -w 1 /sbin/ipvsadm -a -t $WEB_VIP:80 -r $WEB_RIP2:80 -g -w 1 touch /var/lock/subsys/ipvsadm >/dev/null 2>&1 # set Arp /sbin/arping -I eth0 -c 5 -s $WEB_VIP $GW >/dev/null 2>&1 ;; stop) /sbin/ipvsadm -C /sbin/ipvsadm -Z ifconfig eth0:0 down route del $WEB_VIP >/dev/null 2>&1 rm -rf /var/lock/subsys/ipvsadm >/dev/null 2>&1 /sbin/arping -I eth0 -c 5 -s $WEB_VIP $GW echo "ipvsadm stoped" ;; status) if [ ! -e /var/lock/subsys/ipvsadm ];then echo "ipvsadm is stoped" exit 1 else ipvsadm -ln echo "..........ipvsadm is OK." fi ;; *) echo "Usage: $0 {start|stop|status}" exit 1 esac exit 0

b）配置RealServer脚本

#!/bin/bash # Written by Supernic # description: Config realserver lo and apply noarp WEB_VIP=192.168.1.35 . /etc/rc.d/init.d/functions case "$1" in start) ifconfig lo:0 $WEB_VIP netmask 255.255.255.255 broadcast $WEB_VIP /sbin/route add -host $WEB_VIP dev lo:0 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl -p >/dev/null 2>&1 echo "RealServer Start OK" ;; stop) ifconfig lo:0 down route del $WEB_VIP >/dev/null 2>&1 echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce echo "RealServer Stoped" ;; status) # Status of LVS-DR real server. islothere=`/sbin/ifconfig lo:0 | grep $WEB_VIP` isrothere=`netstat -rn | grep "lo:0" | grep $web_VIP` if [ ! "$islothere" -o ! "isrothere" ];then # Either the route or the lo:0 device # not found. echo "LVS-DR real server Stopped." else echo "LVS-DR Running." fi ;; *) # Invalid entry. echo "$0: Usage: $0 {start|status|stop}" exit 1 ;; esac exit 0

或者采用secondary ip address方式配置

vi /etc/sysctl.conf
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
sysctl -p
ip addr add 192.168.1.35/32 dev lo
ip add list \\查看是否绑定

c）启动lvs-dr脚本和RealServer脚本，在DR上可以查看当前状态，（lvs-dr脚本只在测试LVS时使用，后面的调度只用到keepalived就行了，不用此脚本）
watch ipsadm –ln

四． 利用Keepalived实现负载均衡和高可用性

1.在主调度器上配置Keepalived

1. ! Configuration File for keepalived 2. 3. global_defs { 4. notification_email { 5. support@gmail.com 6. } 7. notification_email_from sns-lvs@gmail.com 8. smtp_server 127.0.0.1 9. # smtp_connect_timeout 30 10. router_id LVS_DEVEL 11. } 12. 13. # 20120225 written by :supernic 14. # VIP1 15. vrrp_instance VI_1 { 16. state MASTER #备份服务器上将MASTER改为BACKUP 17. interface eth0 18. virtual_router_id 51 19. priority 100 # 备份服务上将100改为99 20. advert_int 1 21. authentication { 22. auth_type PASS 23. auth_pass 1111 24. } 25. virtual_ipaddress { 26. 192.168.1.35 27. #(如果有多个VIP，继续换行填写.) 28. } 29. } 30. 31. virtual_server 192.168.1.35 80 { 32. delay_loop 6 #(每隔10秒查询realserver状态) 33. lb_algo wrr #(lvs 算法) 34. lb_kind DR #(Direct Route) 35. persistence_timeout 60 #(同一IP的连接60秒内被分配到同一台realserver) 36. protocol TCP #(用TCP协议检查realserver状态) 37. 38. real_server 192.168.1.33 80 { 39. weight 3 #(权重) 40. TCP_CHECK { 41. connect_timeout 10 #(10秒无响应超时) 42. nb_get_retry 3 43. delay_before_retry 3 44. connect_port 80 45. } 46. } 47. real_server 192.168.1.34 80 { 48. weight 3 49. TCP_CHECK { 50. connect_timeout 10 51. nb_get_retry 3 52. delay_before_retry 3 53. connect_port 80 54. } 55. } 56. } 57.

.
2.backup配置同上，只需将state和priority修改即可。

3.停掉lvs-dr脚本，启动keepalived服务

/usr/local/sbin/lvs-dr.sh stop 停止lvs-dr脚本

/etc/init.d/keepalived start 启动keepalived 服务，keepalived就能利用keepalived.conf 配置文件，实现负载均衡和高可用.

整个LVS负均衡HA方案，由keepalived.conf一个文件即可搞定！
chkcofig keepalived on //让keepalvied开机自启动

ipvsadm –ln //查看LVS服务是否正常

[root@localhost ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.1.35:80 wrr persistent 60 -> 192.168.1.33:80 Route 3 0 0 -> 192.168.1.34:80 Route 3 0 0 [root@localhost ~]#

4.tail -f /var/log/message //监听日志，查看状态，测试LVS负载均衡及高可用性是否有效。
5. 停Master服务器的keepalived服务，查看BAKCUP服务器是否能正常接管服务。

附件：http://down.51cto.com/data/2359923