Keepalived+LVS实现负载均衡高可用

安装环境:Centos6.5 x86_64系统最小化安装
实验环境:
LVS1:172.16.35.206
LVS2:172.16.35.81
Nginx1:172.16.35.249
Nginx2:172.16.35.75
VIP:172.16.35.211

实验拓扑图如下:




安装部署过程如下:
一.部署LVS1和LVS2

需要安装LVS软件和keepalived。
脚步如下:
//ipvsadm和iptables不能同时使用，所以这里需要清空iptables的表规则和信息或者关闭iptables
1.安装LVS

#!/bin/bash
yum install ipvsadm -y
/sbin/iptables -F
/sbin/iptables -Z
/sbin/ipvsadm -C

2.安装Keepalived
脚本如下:

#!/bin/bash
yum install kernel-devel gcc gcc-c++ openssl-devel -y
wget http://www.keepalived.org/software/keepalived-1.2.12.tar.gz tar zxvf keepalived-1.2.12.tar.gz
cd keepalived-1.2.12
./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/*/
make && make install
mkdir /etc/keepalived
\cp  -f keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
\cp -f keepalived/etc/init.d/keepalived.init /etc/init.d/keepalived
\cp -f keepalived/etc/init.d/keepalived.sysconfig /etc/sysconfig/keepalived
\cp -f /usr/local/keepalived/sbin/keepalived /sbin/

二:配置Keepalived
LVS1的配置文件如下

! Configuration File for keepalived
global_defs {
notification_email {
445188383@qq.com #定义接受报警邮件的账号
}
notification_email_from root@localhost    #设置发送报警邮件的账号
smtp_server 127.0.0.1 #定义发送邮件的邮箱地址
smtp_connect_timeout 15 #发送邮件的超时时间
router_id LVS        #运行keepalived机器的一个标识信息
}
vrrp_instance VI_1 {    #定义一个VRRP实例
state BACKUP        #设置虚拟路由器的状态,只是一个标识作用，最后还是根据权重来竞选
interface eth0        #绑定虚拟IP的接口
virtual_router_id 51 #虚拟路由ID，同一组keepalived的虚拟路由ID要相同
　　priority 100         #优先级，高的优先级，将会成为master并绑定VIP
advert_int 1         #定义检查间隔
smtp_alert            #当状态切换的时候发送邮件通知
authentication {     #VRRP报文是加密的，这里定义了加密的方式和密码
auth_type PASS
auth_pass 1111
}
virtual_ipaddress { #定义虚拟IP地址
172.16.35.211
}
}
virtual_server 172.16.35.211 80 { #定义一个lvs server实例
delay_loop 6     #定义检查的间隔
lb_algo wlc     #定义使用的lvs算法
lb_kind DR      #定义使用lvs的哪种模型
nat_mask 255.255.255.0 #定义掩码
persistence_timeout 50 #定义会话保持的时间
protocol TCP #定义检查使用的协议
real_server 172.16.35.249 80 { #定义一个lvs的实例
weight 3     #定义服务的权重
TCP_CHECK { #定义使用tcp协议进行检查
connect_timeout 3 #定义检查的超时时间
nb_get_retry 3 #定义检查失败的重试次数
delay_before_retry 3 #定义两次检查的时间间隔
connect_port 80 #定义检查的端口
}
}
real_server 172.16.35.75 80 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}

LVS2的配置文件如下:
将LVS1的的配置文件priority改成比100小的值即可,其他无需改动。

Nginx配置:
配置lVS DR模型:

#!/bin/bash
#
# Script to start LVS DR real server.
# description: LVS DR real server
#
.  /etc/rc.d/init.d/functions
VIP=172.16.35.211
host=`/bin/hostname`
case "$1" in
start)
# Start LVS-DR real server on this machine.
/sbin/ifconfig lo down
/sbin/ifconfig lo up
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:0
;;
stop)
# Stop LVS-DR real server loopback device(s).
/sbin/ifconfig lo:0 down
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
;;
status)
# Status of LVS-DR real server.
islothere=`/sbin/ifconfig lo:0 | grep $VIP`
isrothere=`netstat -rn | grep "lo:0" | grep $VIP`
if [ ! "$islothere" -o ! "isrothere" ];then
# Either the route or the lo:0 device
# not found.
echo "LVS-DR real server Stopped."
else
echo "LVS-DR real server Running."
fi
;;
*)
# Invalid entry.
echo "$0: Usage: $0 {start|status|stop}"
exit 1
;;
esac

安装Nginx

#!/bin/bash
groupadd -r nginx
useradd -r -g nginx nginx
yum install gcc gcc-c++ openssl-devel pcre-devel wget vim automake autoconf -y
wget http://nginx.org/download/nginx-1.4.7.tar.gz #这个地址可能有的时候无法解析到域名下载不到这个包，需要自己去下载包然后安装安装步骤一步一步安装
#wget http://mirror.yongbok.net/nongnu/libunwind/libunwind-1.1.tar.gz wget http://gperftools.googlecode.com/files/gperftools-2.1.tar.gz #                       libunwind install
tar -xvf libunwind-1.1.tar.gz
cd libunwind-1.1
CFLAGS=-fPIC ./configure
make CFLAGS=-fPIC
make CFLAGS=-fPIC install
cd ..
#                       gperftools install
tar -xvf gperftools-2.1.tar.gz
cd gperftools-2.1
./configure
make && make install
echo "/usr/local/lib" > /etc/ld.so.conf.d/usr_local_lib.conf
ldconfig
cd ..
#                       nginx install
tar zxvf nginx-1.4.7.tar.gz
cd nginx-1.4.7
./configure --prefix=/usr/local/nginx \
--with-http_stub_status_module \
--with-http_gzip_static_module \
--with-google_perftools_module \
--user=nginx \
--group=nginx \
--with-http_ssl_module \
--with-pcre
make && make insatall
#                       Setup
mkdir /tmp/tcmalloc
chmod 0777 /tmp/tcmalloc
#简单的一个nginx优化
cat >> /etc/sysctl.conf <<EOF
net.ipv4.tcp_max_tw_buckets = 6000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_syncookies = 1
net.core.somaxconn = 262144
net.core.netdev_max_backlog = 262144
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_keepalive_time = 30
EOF

测试:
分别给Nginx1和Nginx2提供测试页面:
echo "172.16.35.75" > /usr/local/nginx/html/index.html
echo "172.16.35.249" > /usr/local/nginx/html/index.html




测试高可用功能:
关闭LVS1的keepalived功能，测试








到此一个比较成熟的负载均衡高可用方案到此结束。下次会介绍keepalived+nginx的负载均衡高可用