LVS+KEEPALIVED实现负载均高可用集群

LVS+KEEPALIVED负载均衡高可用集群

通过命令检测kernel是否已经支持LVS的ipvs模块
[root@www ~]# modprobe -l|grep ipvs
kernel/net/netfilter/ipvs/ip_vs.ko
kernel/net/netfilter/ipvs/ip_vs_rr.ko
kernel/net/netfilter/ipvs/ip_vs_wrr.ko
kernel/net/netfilter/ipvs/ip_vs_lc.ko
kernel/net/netfilter/ipvs/ip_vs_wlc.ko
kernel/net/netfilter/ipvs/ip_vs_lblc.ko
kernel/net/netfilter/ipvs/ip_vs_lblcr.ko
kernel/net/netfilter/ipvs/ip_vs_dh.ko
kernel/net/netfilter/ipvs/ip_vs_sh.ko
kernel/net/netfilter/ipvs/ip_vs_sed.ko
kernel/net/netfilter/ipvs/ip_vs_nq.ko
kernel/net/netfilter/ipvs/ip_vs_ftp.ko
有蓝色的两项表明系统内核已经默认支持IPVS
下面就可以进行安装
两种方式
第一：源码包安装
Make
Make install
可能会出错
由于编译程序找不到对应的内核
解决：ln -s /usr/src/kernels/2.6.32-279.el6.x86_64/ /usr/src/linux
第二种：
Rpm包安装
红帽的LoadBalancer是有这个包的
[root@www ~]# yum list |grep ipvs
ipvsadm.x86_64 1.25-10.el6 LoadBalancer

[root@www ~]# yum install ipvsadm.x86_64
Loaded plugins: product-id, subscription-manager
Updating certificate-based repositories.
Unable to read consumer identity
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package ipvsadm.x86_64 0:1.25-10.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=========================================================================================================================================
Package Arch Version Repository Size
=========================================================================================================================================
Installing:
ipvsadm x86_64 1.25-10.el6 LoadBalancer 41 k

Transaction Summary
=========================================================================================================================================
Install 1 Package(s)

Total download size: 41 k
Installed size: 74 k
Is this ok [y/N]: y
Downloading Packages:
ipvsadm-1.25-10.el6.x86_64.rpm | 41 kB 00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installed products updated.
Verifying: ipvsadm-1.25-10.el6.x86_64 1/1

Installed:
ipvsadm.x86_64 0:1.25-10.el6 &
3ff0
nbsp;

Complete!
[root@www ~]# ipvsadm --help
ipvsadm v1.25 2008/5/15 (compiled with popt and IPVS v1.2.1)
Usage:
ipvsadm -A|E -t|u|f service-address [-s scheduler] [-p [timeout]] [-M netmask]
ipvsadm -D -t|u|f service-address
ipvsadm -C
ipvsadm -R
ipvsadm -S [-n]
ipvsadm -a|e -t|u|f service-address -r server-address [options]
ipvsadm -d -t|u|f service-address -r server-address
ipvsadm -L|l [options]
ipvsadm -Z [-t|u|f service-address]
ipvsadm --set tcp tcpfin udp
ipvsadm --start-daemon state [--mcast-interface interface] [--syncid sid]
ipvsadm --stop-daemon state
如果看到提示表明IPVS已经安装成功
开始配置LVS两种方式
Ipvsadmin和prianha来进行配置
第一种ipvsadmin
638 ipvsadm -A -t 192.168.1.250:80 -s rr -p 600添加一个虚拟ip采用轮询的方式每个节点保持10分钟
639 ipvsadm -a -t 192.168.1.250:80 -r 192.168.1.200:80 -g添加real server -g表示DR方式
640 ipvsadm -a -t 192.168.1.250:80 -r 192.168.1.201:80 -g添加real server -g表示DR方式
641 ifconfig eth0:0 192.168.1.250 netmask 255.255.255.0 up 添加虚拟ip
643 route add -host 192.168.1.250 dev eth0:0添加虚拟ip路由
645 echo "1" > /proc/sys/net/ipv4/ip_forward添加路由转发，Dr不是必须的NAT是必须的
[root@www ~]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.1.250:http rr
-> www.ha1.com:http Route 1 0 0
-> www.ha2.com:http Route 1 0 0
方便管理的话可以做一个启动和停止脚本
#!/bin/sh
# description: Start LVS of Director server
VIP=192.168.1.250
RIP1=192.168.1.200
RIP2=192.168.1.201
./etc/rc.d/init.d/functions
case "$1" in
start)
echo " start LVS of Director Server"
# set the Virtual IP Address and sysctl parameter
/sbin/ifconfig eth0:0 $VIP broadcast $VIP netmask 255.255.255.255 up
echo "1" >/proc/sys/net/ipv4/ip_forward
#Clear IPVS table
/sbin/ipvsadm -C
#set LVS
/sbin/ipvsadm -A -t $VIP:80 -s rr -p 600
/sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -g
/sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -g
#Run LVS
/sbin/ipvsadm
;;
stop)
echo "close LVS Directorserver"
echo "0" >/proc/sys/net/ipv4/ip_forward
/sbin/ipvsadm -C
/sbin/ifconfig eth0:0 down
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac

第二种方式采用piranha来配置lvs,也可以采用网页登陆piranha方式进行配置
首先安装piranha
yum install piranha
piranha安装完毕后,会产生/etc/sysconfig/ha/lvs.cf文件
编辑这个文件
[root@www ha]# vim /etc/sysconfig/ha/lvs.cf
service = lvs
primary = 192.168.1.202
backup = 0.0.0.0
backup_active = 0
keepalive = 6
deadtime = 10
debug_level = NONE
network = direct
virtual server1 {
address = 192.168.1.250 eth0:0
# vip_nmask = 255.255.255.255
active = 1
load_monitor = none
timeout = 5
reentry = 10
port = http
send = "GET / HTTP/1.0\r\n\r\n"
expect = "HTTP"
scheduler = rr
protocol = tcp
# sorry_server = 127.0.0.1

server Real1 {
address = 192.168.1.200
active = 1
weight = 1
}

server Real2 {
address = 192.168.1.201
active = 1
weight = 1
}
}
[root@www ha]# /etc/init.d/pulse status
pulse (pid 3211) is running...
[root@www ha]# cat /proc/sys/net/ipv4/ip_forward
1
[root@www ha]# route add -host 192.168.1.250 dev eth0:0

Relserver配置
VIP=192.168.1.250
/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:0
echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p
防止arp
验证





[root@www ha]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.1.250:http rr
-> www.ha1.com:http Route 1 0 5
-> www.ha2.com:http Route 1 0 5
当我们关掉一台节点的httd进程时



机子会把这台节点提出去



这时候就只会访问ha1的站点了



形成了负载均衡
剩下的就是对这两台LVS做keepalived高可用 也就是使用vrrp
安装keepalived
[root@www keepalived]# rpm -Uvh ~/keepalived-1.2.7-5.x86_64.rpm
配置keepalived.conf
vrrp_instance VI_1 {
state MASTER # master
interface eth0
lvs_sync_daemon_interface eth1
virtual_router_id 50 # unique, but master and backup is same
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass uplooking
}
virtual_ipaddress {
192.168.1.250 # vip
}
}
virtual_server 192.168.1.250 80 {
delay_loop 20
lb_algo rr
lb_kind DR
nat_mask 255.255.255.0
protocol TCP

real_server 192.168.1.200 80 {
weight 1
TCP_CHECK {
connect_timeout 3
}
}
real_server 192.168.1.201 80 {
weight 1
TCP_CHECK {
connect_timeout 3
}
}
}
完整验证：
第一步：keepalived节点down掉一台



从上图中可以看出虚拟ip已经不在这台ma1上面
下面我们来看ma2的配置



从上图中可以看出ma2的各项进程都运行正常，虚拟ip迁移，实现了LVS的高可用





可以看出依然实现了负载均衡
现在故障处理完毕启动ma1的keepalived



虚拟ip又回到ha1了

下面我们来看一下ha1，停掉ha1上httpd服务
查看ma1



我们可以看到虚拟IP没有移动，但是在lvs中的ha1的路由表已经被踢出来了
下面我们来看ma2



也可以看到虚拟IP没有移动过来，但是在lvs中的ha1的路由表已经被踢出来了
现在访问web



就只能看到ha2的web页面了
现在启动ha1的web服务



我们可以看到ma1和ma2中ha1的路由表又自动添加进来了





访问web页面





验证完成
总结说明
网络结构



集群说明：
1.使用lvs构成web的负载均衡高可用
2.使用keepalived形成lvs节点的高可用
验证说明：
1.停掉任意节点的keepalived进程虚拟IP会迁移到另外一台去，而web服务器任然是两台进行负载均衡，从而实现lvs节点的高可用
2.停掉任意一台web服务器，虚拟ip不会迁移，但是在lvs配置会把ha1的路由表踢出去从而实现web服务器的高可用