借助keepalived实现高可用haproxy集群

一、环境准备
系统版本：CentOS-6.6 kernel:2.6.32-504.el6.x86_64



两个http节点：
172.16.113.13:80
172.16.113.14:80
两个haproxy节点：
172.16.13.13
172.16.13.14
keepalived双主vip：
172.16.13.1
172.16.13.2

二、网络拓扑




三、httpd安装与测试

yum install httpd -y
vim /var/www/html/index.html
<h1>node1</h1>
service httpd start

四、安装与配置haproxy

yum install haproxy -y

vim /etc/haproxy/haproxy.cfg
global
log         127.0.0.1 local2
chroot      /var/lib/haproxy
pidfile     /var/run/haproxy.pid
maxconn     4000
user        haproxy
group       haproxy
daemon
stats socket /var/lib/haproxy/stats
defaults
mode                    http
log                     global
option                  httplog
option                  dontlognull
option http-server-close
option forwardfor       except 127.0.0.0/8
option                  redispatch
retries                 3
timeout http-request    10s
timeout queue           1m
timeout connect         10s
timeout client          1m
timeout server          1m
timeout http-keep-alive 10s
timeout check           10s
maxconn                 3000
frontend  main *:80
default_backend             websrvs
backend websrvs
balance     roundrobin
server  web1 172.16.113.13:80 maxconn 5000
server  web1 172.16.113.14:80 maxconn 5000
listen stats *:9103
stats enable

service haproxy start

五、安装配置keepalived
两节点时间同步：

ntpdate -u 172.16.0.1
crontab -e
*/3 * * * * /usr/sbin/ntpdate 172.16.0.1 &> /dev/null

基于主机名互访：

vim /etc/hosts
172.16.13.13        node1.magedu.com  node1
172.16.13.14        node2.magedu.com  node2

基于密钥认证：

ssh-keygen -t rsa -f /root/.ssh/id_rsa -P ''
ssh-copy-id -i /root/.ssh/id_rsa.pub root@HOSTNAME

安装配置keepalived

yum install keepalived -y
vim /etc/keepalived/keepalived.conf

节点1：172.16.13.13 上配置文件

global_defs {
notification_email {
admin@126.com
}
notification_email_from root@node3.wuhf.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_down {
script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
interval 2
weight -10
}
vrrp_script chk_haproxy {
script "killall -0 haproxy &> /dev/null"
interval 1
weight -5
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 62
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 21111
}
virtual_ipaddress {
172.16.13.1/32 brd 172.16.13.1 dev eth0 label eth0:0
}
track_script {
chk_down
chk_haproxy
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 63
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 2b1111
}
virtual_ipaddress {
172.16.13.2/32 brd 172.16.13.2 dev eth0 label eth0:1
}
track_script {
chk_down
chk_haproxy
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}

节点2：172.16.13.14 上配置文件

global_defs {
notification_email {
admin@126.com
}
notification_email_from root@node3.wuhf.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_down {
script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
interval 2
weight -10
}
vrrp_script chk_haproxy {
script "killall -0 haproxy &> /dev/null"
interval 1
weight -5
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 62
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 21111
}
virtual_ipaddress {
172.16.13.1/32 brd 172.16.13.1 dev eth0 label eth0:0
}
track_script {
chk_down
chk_haproxy
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 63
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 2b1111
}
virtual_ipaddress {
172.16.13.2/32 brd 172.16.13.2 dev eth0 label eth0:1
}
track_script {
chk_down
chk_haproxy
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}

创建keepalived的辅助脚本notify.sh

vim /etc/keepalived/notify.sh
vip=172.16.13.1
contact='kaadmin@localhost'
notify() {
mailsubject="`hostname` to be $1: $vip floating"
mailbody="`date '+%F %H:%M:%S'`: vrrp transition, `hostname` changed to be $1"
echo $mailbody | mail -s "$mailsubject" $contact
}
case "$1" in
master)
/etc/rc.d/init.d/haproxy start
notify master
exit 0
;;
backup)
/etc/rc.d/init.d/haproxy restart
notify backup
exit 0
;;
fault)
/etc/rc.d/init.d/haproxy stop
notify fault
exit 0
;;
*)
echo 'Usage: `basename $0` {master|backup|fault}'
exit 1
;;
esac

chmod +x /etc/keepalived/notify.sh
service keepalived start; ssh node2 'service keepalived start'

六、测试高可用

touch /etc/keepalived/down
#创建down文件，测试vrrp_script chk_down函数是不是生效，如果生效此节点权重值会降低，另一个节点会将backup状态的vip转变为master状态

service keepalived stop
#关掉keepalived进程，查看vip会不会转变

service haproxy stop;service httpd start
#强制httpd进程抢占haproxy进程的80端口，查看ip地址会不会转变

七、总结
keppalived的高可用有一种情况下是会发生错乱的
双主模式下，一个节点宕机（eth0这个网络接口down掉的情况下），此时这个节点上的vip并不会自动转移到另一个节点上，使用tcpdump抓包分析，可以看出虽然eth0接口down了，但是eth0:0或eth0:1还存在，节点可以通过这两个别名接口接收广播报文，但是无法响应报文；此时另一个节点会不断的发出权重比配消息，但是不会自动判断对方节点是否可用，所以不会自动加载down掉的VIP

本文出自 “铜锣湾” 博客，请务必保留此出处http://wuhf2015.blog.51cto.com/8213008/1662898