lvs+keepalived

[align=center]Lvs+keepalived[/align]
前期为了测试云存储搭建了一个lvs的tomcat负载均衡集群，近期要求的云存储以及前端集群做破坏性试验，这样为了满足系统的高可用性对lvs的director节点做高可用，以及对hadoop的管理节点也要做高可用，因为公司测试环境及线上环境linux系统版本较老，heartbeat2与heartbeat3等版本可能安装稍微麻烦一点，这里对lvs的director节点尝试用keepalived做高可用，对hadoop用heartbeat做高可用（以及共享存储，本文暂时不介绍dahoop的高可用及共享存储，以及安装配置测试遇到的“报错”和“问题”，过一段时间补上）,下面介绍lvs负载均衡集群前端分发负载director节点的keepalived高可用实现过程：
目前架构图如下：




1、软件下载安装：（director主节点与备用节点都安装）
1）lvs软件ipvsadm的下载安装
#wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
#tar zxvf ipvsadm-1.24.tar.gz
#ln -s /usr/src/kernels/2.6.9-89.EL-smp-x86_64 /usr/src/linux //注释：这一步针对自己的系统做，否则一会安装ipvsadm会报错
# cd ipvsadm-1.24
#make
#make install
2）keepalived的下载安装
#wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz
# tar zxvf keepalived-1.1.15.tar.gz
#cd keepalived-1.1.15
//注释：如果系统rhel版本低于5.0以下安装报错，这里要修改系统配置
//vi /usr/src/linux/include/linux/types.h将下面2行用/* */注释掉，如下图
/*
typedef __u16 __bitwise __sum16;
typedef __u32 __bitwise __wsum;
*/
#./configure
#make
#make install
2、将keepalived做成服务启动
#cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/注释:拷贝服务启动文件
#cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
#mkdir /etc/keepalived/注释：创建配置文件目录
#cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/注释：拷贝配置文件
#cp /usr/local/sbin/keepalived /usr/sbin/注释：拷贝可执行文件
#service keepalived start注释:测试服务启动与关闭
#service keepalived stop
3、配置keepalived
在配置keepalived时发现这个软件用着比heartbeat要简单很多
主节点的配置
#vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
reaaa@163.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.2.120
}
}
virtual_server 192.168.2.120 8080 {
delay_loop 6
lb_algo wlc
lb_kind DR
persistence_timeout 1
protocol TCP
real_server 192.168.5.54 8080 {
weight 1
TCP_CHECK {
connect_timeout 1
nb_get_retry 3
delay_before_retry 3
connect_port 8080
}
}
real_server 192.168.5.57 8080 {
weight 1
TCP_CHECK {
connect_timeout 1
nb_get_retry 3
delay_before_retry 3
connect_port 8080
}
}
}
BACKUP节点配置：
#vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
renxiaotian@ninetowns.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.2.120
}
}
virtual_server 192.168.2.120 8080 {
delay_loop 6
lb_algo wlc
lb_kind DR
persistence_timeout 1
protocol TCP
real_server 192.168.5.54 8080 {
weight 1
TCP_CHECK {
connect_timeout 1
nb_get_retry 3
delay_before_retry 3
connect_port 8080
}
}
real_server 192.168.5.57 8080 {
weight 1
TCP_CHECK {
connect_timeout 1
nb_get_retry 3
delay_before_retry 3
connect_port 8080
}
}
}
注释：因为我用的tomcat后端8080端口没有修改为80，这个请读者自己按照自己的情况配置，包括各种IP与持久连接，realserver的数量，如果keepalived启动后查看后端的realserver没有达到自己配置的数量，那么就是端口没有指对。
3、配置realserver
#vi realserver.sh在脚本中添加如下内容
#!/bin/bash
VIP=192.168.2.120
. /etc/rc.d/init.d/functions
case "$1" in
start)
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:0
sysctl -p > /dev/null 2>&1
echo "realserver start OK"
;;
stop)
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 down
/sbin/route del $VIP > /dev/null 2>&1
echo "realserver stoped"
#chmod +x realserver.sh
并执行
#./realserver.sh
注释：2台web服务器做相同操作
如果想开机启动配置rc.local
#echo ‘/root/realserver.sh start >> /etc/rc.local
Keepalived的日志在/var/log/messages故障切换测试可以查看
watch -n 1 'ipvsadm -L -n'连续查看lvs状况命令