Haproxy+keepalived TCP 层的负载均衡

一、安装环境准备：

l [root@slave3 ~]# yum install ipvsadm kernel-developenssl-devel popt-devel libnl-devel gcc make –y

l Makebinding to non-local Virtual IPs on all nodes

vi /etc/sysctl.conf 在各haproxy的机器上，在/etc/sysctl.conf下添加如下设置

net.ipv4.ip_nonlocal_bind = 1

否则：会出现 scoket cannot connect xxxx.xxxx.xxxx.xxxx:port

二、背景介绍

Keepalived是一个基于VRRP协议来实现的WEB
服务高可用方案，可以利用其来避免单点故障。一个WEB服务至少会有2台服务器运行Keepalived，一台为主服务器（MASTER），一台为备份服务器（BACKUP），但是对外表现为一个虚拟IP，主服务器会发送特定的消息给备份服务器，当备份服务器收不到这个消息的时候，即主服务器宕机的时候，备份服务器就会接管虚拟IP，继续提供服务，从而保证了高可用性

HAProxy介绍

反向代理服务器,支持双机热备支持虚拟主机, ,拥有非常不错的服务器健康检查功能,当其代理的后端服务器出现故障,
HAProxy会自动将该服务器摘除,故障恢复后再自动将该服务器加入,
新的1.3引入了frontend,backend；frontend根据任意
HTTP请求头内容做规则匹配,然后把请求定向到相关的backend

三、安装keepalived
与haproxy

在

[root@slave3 ~]# yum -y install haproxy keepalived

四、配置haproxy

[root@slave3 ~]# cd /etc/haproxy/

[root@slave3 haproxy]# cp haproxy.cfg haproxy.cfg.orig

[root@slave3 haproxy] vi haproxy.cfg

global

log 127.0.0.1 local2

chroot /var/lib/haproxy

pidfile /var/run/haproxy.pid

maxconn 4000

user haproxy

group haproxy

daemon

#turn onstats unix socket

stats socket/var/lib/haproxy/stats

defaults

mode tcp

log global

option dontlognull

option http-server-close

option redispatch

option abortonclose

retries 3

timeout queue 1m

timeoutconnect 10s

timeoutclient 1m

timeoutserver 1m

timeoutcheck 10s

maxconn 3000

listen proxy_apalad

bind192.168.211.200:8082

mode tcp

option httpchk

balanceroundrobin

server slave3192.168.211.185:21000 weight 1 maxconn 1000

server slave4192.168.211.253:21000 weight 1 maxconn 1000

listen admin_stats

bind192.168.211.200:8081

mode http

optionhttplog

maxconn 10

stats refresh30s

stats uri/stats

备注：加日志支持

1. 编辑/etc/rsyslog.conf在最下边增加

local3.* /var/log/haproxy.log

local0.* /var/log/haproxy.log

local2.* /var/log/haproxy.log

2. 编辑 /etc/sysconfig/rsyslog修改：

SYSLOGD_OPTIONS="-r -m 0"

3. 重启日志服务

#service rsyslog restart

五、配置keepalived

! Configuration File for keepalived

vrrp_script chk_http_port {

script"/etc/keepalived/check_haproxy.sh"

interval 2

weight 2

}

#vrrp_script chk_haproxy{

# script"killall -0 haproxy"

# interval 2

# weight -2

#}

vrrp_instance VI_1 {

state MASTER

interfaceeth1

virtual_router_id 51

priority 100

advert_int 1

authentication {

auth_type PASS

auth_pass 1111

}

virtual_ipaddress {

192.168.211.200

}

track_script{

#chk_haproxy

chk_http_port

}

#notify_master "/etc/keepalived/scripts/start_haproxy.sh"

#notify_backup "/etc/keepalived/scripts/stop_keepalived.sh"

#notify_fault "/etc/keepalived/scripts/stop_keepalived.sh"

#notify_stop "/etc/keepalived/scripts/stop_haproxy.sh"

配置keepalived 备用节点，与主配节点的区别有两处

Ø 将state MASTER 修改为state BACKUP

Ø 将priority 100 修改为priority 99

六、附加脚本

[root@slave3 keepalived]# cd /etc/keepalived/

[root@slave3 keepalived]# vi check_haproxy.sh

#!/bin/bash

if [ $(ps -C haproxy --no-header | wc-l) -eq 0 ]; then

/etc/init.d/haproxy start

fi

sleep 2

if [ $(ps -C haproxy --no-header | wc-l) -eq 0 ]; then

/etc/init.d/keepalived stop

fi

修改脚本执行权限:

[root@slave3 keepalived]# chmod +x /etc/keepalived/check_haproxy.sh

[root@slave3 keepalived]# vi /etc/keepalived/script/start_haproxy.sh

#!/bin/bash

sleep 5

get=`ip addr |grep 192.168.211.200 |wc -l`

echo $get >>/etc/keepalived/scripts/start_ha.log

if [ $get -eq 1 ]

then

echo"`date +%c` success to get vip" >> /etc/keepalived/scripts/start_ha.log

/etc/init.d/haproxy start

else

echo"`date +%c` can not get vip" >>/etc/keepalived/scripts/start_ha.log

fi

[root@slave3 keepalived]# vi /etc/keepalived/script/stop_haproxy.sh

/etc/init.d/haproxystop

[root@slave3 keepalived]# vi /etc/keepalived/script/stop_keepalived.sh

#!/bin/bash

pid=`pidof keepalived`

if [ "$pid"x == ""x ]

then

echo"`date +%c` no keepalived process id" >> /etc/keepalived/scripts/stop_keep.log

else

echo"`date +%c` will stop keepalived " >> /etc/keepalived/scripts/stop_keep.log

/etc/init.d/keepalived stop

fi

/etc/init.d/keepalived stop

七、测试

启动haproxy: [root@slave4~]# /etc/init.d/haproxy start

启动keepalived：[root@slave3 scripts]# /etc/init.d/keepalived start

八、参考文档

http://blog.laimbock.com/2014/10/01/howto-setup-high-available-haproxy-with-keepalived/

Howtosetup High-Available HAProxy with Keepalived

/article/2453073.html

haproxy做TCP层的负载均衡

http://my.oschina.net/davehe/blog/162560

keepalived+haproxy高可用负载均衡

http://blog.liuts.com/post/223/

基于Keepalived+Haproxy搭建四层负载均衡器[原创]