您的位置:首页 > 其它

部署Keepalived+LVS

2015-10-12 22:22 281 查看
部署keepalived高可用
部署环境:

KEEP1 IP :10.0.0.5
KEEP2 IP : 10.0.0.6

两台服务器的环境都是如下
[root@KEEP2 ~]# cat /etc/redhat-release
CentOS release 6.6 (Final)
[root@KEEP2 ~]# uname -v
#1 SMP Wed Jul 15 10:13:09 UTC 2015
[root@KEEP2 ~]# uname -a
Linux KEEP2 2.6.32-504.30.3.el6.x86_64 #1 SMP Wed Jul 15 10:13:09 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@KEEP2 ~]# uname -r
2.6.32-504.30.3.el6.x86_64

下面是正式开始部署Keepalived,在KEEP1和KEEP2做相同的操作,知道第四部,配置配置文件才有所差别

#####################################################################################################################################################################################################################做相关内核软连接

安装keepalived之前要先做软连接

ls /usr/src/kernels/`uname -r` -ld
ln -s /usr/src/kernels/`uname -r` /usr/src/linux
ls /usr/src/ -l
=========================================================================
[root@KEEP2 ~]# ls /usr/src/kernels/`uname -r` -ld
drwxr-xr-x. 22 root root 4096 Jul 20 10:47 /usr/src/kernels/2.6.32-431.el6.x86_64
[root@KEEP2 ~]# ln -s /usr/src/kernels/`uname -r` /usr/src/linux
[root@KEEP2 ~]# ls /usr/src/ -l
total 8
drwxr-xr-x. 2 root root 4096 Sep 23 2011 debug
drwxr-xr-x. 4 root root 4096 Jul 21 09:16 kernels
lrwxrwxrwx 1 root root 38 Aug 18 08:29 linux -> /usr/src/kernels/2.6.32-431.el6.x86_64
[root@KEEP2 ~]#

#######################################################################
#######################################################################
#######################################################################
操作步骤2之前可以需要安装下面依赖库
yum install openssl* -y
yum install libnl* -y

2.下载并安装keepalived软件
[ ! -d /home/lvnian/tools ] && mkdir /home/lvnian/tools -p
cd /home/lvnian/tools
wget http://www.keepalived.org/software/keepalived-1.2.10.tar.gz tar xf keepalived-1.2.10.tar.gz
cd keepalived-1.2.10
./configure
make && make install
======================================================
安装简略过程
[root@KEEP2 keepalived-1.2.10]# ./configure

config.status: creating keepalived/libipvs-2.6/Makefile

Keepalived configuration
------------------------
Keepalived version : 1.2.10
Keepalived configuration
------------------------
Keepalived version : 1.2.10
Compiler : gcc
Compiler flags : -g -O2 -DFALLBACK_LIBNL1
Extra Lib : -lssl -lcrypto -lcrypt -lnl
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
IPVS use libnl : Yes (如果这个为no 可以选择yum install libnl* -y 在编译安装)
Use VRRP Framework : Yes
Use VRRP VMAC : Yes
SNMP support : No
SHA1 support : No
Use Debug flags : No

[root@KEEP2 keepalived-1.2.10]# make && make install
make[1]: Leaving directory `/home/deng/tools/keepalived-1.2.10/keepalived'
make -C genhash install
make[1]: Entering directory `/home/deng/tools/keepalived-1.2.10/genhash'
install -d /usr/local/bin
install -m 755 ../bin/genhash /usr/local/bin/
install -d /usr/local/share/man/man1
install -m 644 ../doc/man/man1/genhash.1 /usr/local/share/man/man1
make[1]: Leaving directory `/home/deng/tools/keepalived-1.2.10/genhash'
[root@KEEP2 keepalived-1.2.10]#

编译参数说明:
--with-kernel-dir=DIR 这个参数很重要,但是这个参数并不是要把keepalived编译到内核中,而是指定使用的内核源码里面的头文件,就是include目录,如果使用LVS,才需要这个参数,但是就算使用了LVS也可以不需要这个参数,但需要做软连接:
ln -s /usr/src/kernels/`uname -r` /usr/src/linux

#######################################################################
#######################################################################
#######################################################################

3.配置启动规范
/bin/cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
/bin/cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
/bin/cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
/bin/cp /usr/local/sbin/keepalived /usr/sbin/
/etc/init.d/keepalived start
ps -ef |grep keepalived
=================================
[root@KEEP2 keepalived-1.2.10]# /bin/cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
[root@KEEP2 keepalived-1.2.10]# /bin/cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
/etc/init.d/keepalived start
[root@KEEP2 keepalived-1.2.10]# mkdir /etc/keepalived
ps -ef |grep keepalived
[root@KEEP2 keepalived-1.2.10]# /bin/cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
[root@KEEP2 keepalived-1.2.10]# /bin/cp /usr/local/sbin/keepalived /usr/sbin/
[root@KEEP2 keepalived-1.2.10]# /etc/init.d/keepalived start
Starting keepalived: [ OK ]
[root@KEEP2 keepalived-1.2.10]# ps -ef |grep keepalived
root 2765 1 0 06:04 ? 00:00:00 keepalived -D
root 2768 1348 0 06:04 pts/0 00:00:00 grep keepalived
root 2769 2765 0 06:04 ? 00:00:00 keepalived -D
root 2770 2765 0 06:04 ? 00:00:00 keepalived -D
[root@KEEP2 keepalived-1.2.10]#
安装配置成功

上面的配置需要在KEEP1和KEEP2中都执行
#######################################################################
#######################################################################
#######################################################################

4.配置keepalived单实例
[root@KEEP1 keepalived]# pwd
/etc/keepalived
主配置文件如下MASTER:
[root@KEEP1 keepalived]# cat keepalived.conf
! Configuration File for keepalived

global_defs {
notification_email {
759685538@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_1 ##路由ID,主备必须要不相同
}

vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 55 ##虚拟路由ID,主备必须要相同
priority 150 ##主的优先级必须比备的大
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.100/24
10.0.0.200/25
}
}
[root@KEEP1 keepalived]#

备机配置文件BACKUP:
[root@KEEP2 keepalived]# cat keepalived.conf
! Configuration File for keepalived

global_defs {
notification_email {
759685538@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_2
}

vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 55
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.100/24
10.0.0.200/25
}
}
[root@KEEP2 keepalived]#

查看VIP情况
KEEP1:




KEEP2



VIP会跟着主备服务器的切换而漂移
把KEEP1的keepalived服务停止
[root@KEEP1 ~]# /etc/init.d/keepalived stop
Stopping keepalived: [ OK ]

KEEP1



KEEP2




#####################################################################################################################################################################################################################

5.keepalived的日志信息
默认情况下keepalived的日志文件在/var/log/message
[root@KEEP1 ~]# tail -f /var/log/messages
Aug 18 09:06:49 LVS1 Keepalived[6046]: Starting Keepalived v1.2.10 (08/18,2015)
Aug 18 09:06:49 LVS1 Keepalived[6047]: Starting Healthcheck child process, pid=6049
Aug 18 09:06:49 LVS1 Keepalived[6047]: Starting VRRP child process, pid=6050
Aug 18 09:06:49 LVS1 Keepalived_vrrp[6050]: Netlink reflector reports IP 10.0.0.3 added
Aug 18 09:06:49 LVS1 Keepalived_healthcheckers[6049]: Netlink reflector reports IP 10.0.0.3 added
Aug 18 09:06:49 LVS1 Keepalived_vrrp[6050]: Netlink reflector reports IP fe80::20c:29ff:fea7:2f91 added
Aug 18 09:06:49 LVS1 Keepalived_vrrp[6050]: Registering Kernel netlink reflector
Aug 18 09:06:49 LVS1 Keepalived_vrrp[6050]: Registering Kernel netlink command channel
Aug 18 09:06:49 LVS1 Keepalived_vrrp[6050]: Registering gratuitous ARP shared channel
Aug 18 09:06:49 LVS1 Keepalived_healthcheckers[6049]: Netlink reflector reports IP fe80::20c:29ff:fea7:2f91 added
Aug 18 09:06:49 LVS1 Keepalived_healthcheckers[6049]: Registering Kernel netlink reflector
Aug 18 09:06:49 LVS1 Keepalived_healthcheckers[6049]: Registering Kernel netlink command channel
#######################################################################
配置keepalived日志信息到单独文件/var/log/keepalived.log
默认情况如下
[root@KEEP1 keepalived]# cat /etc/sysconfig/keepalived
# Options for keepalived. See `keepalived --help' output and keepalived(8) and
# keepalived.conf(5) man pages for a list of all options. Here are the most
# common ones :
#
# --vrrp -P Only run with VRRP subsystem.
# --check -C Only run with Health-checker subsystem.
# --dont-release-vrrp -V Dont remove VRRP VIPs & VROUTEs on daemon stop.
# --dont-release-ipvs -I Dont remove IPVS topology on daemon stop.
# --dump-conf -d Dump the configuration data.
# --log-detail -D Detailed log messages.
# --log-facility -S 0-7 Set local syslog facility (default=LOG_DAEMON)
#

KEEPALIVED_OPTIONS="-D"(把这个注销换成下面内容)

[root@KEEP1 keepalived]#
#######################################################################
修改后的内容
[root@KEEP1 keepalived]# cat /etc/sysconfig/keepalived
# Options for keepalived. See `keepalived --help' output and keepalived(8) and
# keepalived.conf(5) man pages for a list of all options. Here are the most
# common ones :
#
# --vrrp -P Only run with VRRP subsystem.
# --check -C Only run with Health-checker subsystem.
# --dont-release-vrrp -V Dont remove VRRP VIPs & VROUTEs on daemon stop.
# --dont-release-ipvs -I Dont remove IPVS topology on daemon stop.
# --dump-conf -d Dump the configuration data.
# --log-detail -D Detailed log messages.
# --log-facility -S 0-7 Set local syslog facility (default=LOG_DAEMON)
#

#KEEPALIVED_OPTIONS="-D"
KEEPALIVED_OPTIONS="-D -d -S 0"

[root@KEEP1 keepalived]#
#######################################################################配置/etc/rsyslog.conf cp /etc/rsyslog.conf /etc/rsyslog.conf.oriecho "local0.* /var/log/keepalived.log" >>/etc/rsyslog.conftail -4 /etc/rsyslog.conf =================================[root@KEEP1 etc]# tail -4 /etc/rsyslog.conf
#*.* @@remote-host:514
# ### end of the forwarding rule ###
#keepalived
local0.* /var/log/keepalived.log
[root@KEEP1 etc]#

#######################################################################
#######################################################################
#######################################################################

重启日志服务,检查配置结果
/etc/init.d/rsyslog restarttail /var/log/keepalived.log -f==================
[root@KEEP1 ~]# /etc/init.d/rsyslog restart
Shutting down system logger: [ OK ]
Starting system logger: [ OK ]
[root@KEEP1 ~]# tail /var/log/keepalived.log -f
Aug 18 09:36:07 KEEP1 Keepalived_healthcheckers[14792]: Netlink reflector reports IP 10.0.0.3 added
Aug 18 09:36:07 KEEP1 Keepalived_healthcheckers[14792]: Netlink reflector reports IP fe80::20c:29ff:fea7:2f91 added
Aug 18 09:36:07 KEEP1 Keepalived_healthcheckers[14792]: Registering Kernel netlink reflector
Aug 18 09:36:07 KEEP1 Keepalived_healthcheckers[14792]: Registering Kernel netlink command channel
Aug 18 09:36:07 KEEP1 Keepalived_vrrp[14793]: Netlink reflector reports IP 10.0.0.3 added
Aug 18 09:36:07 KEEP1 Keepalived_vrrp[14793]: Netlink reflector reports IP fe80::20c:29ff:fea7:2f91 added
Aug 18 09:36:07 KEEP1 Keepalived_vrrp[14793]: Registering Kernel netlink reflector
Aug 18 09:36:07 KEEP1 Keepalived_vrrp[14793]: Registering Kernel netlink command channel
Aug 18 09:36:07 KEEP1 Keepalived_vrrp[14793]: Registering gratuitous ARP shared channel
Aug 18 09:36:07 KEEP1 Keepalived_vrrp[14793]: Initializing ipvs 2.6

如果没看到日志内容,但也不提示错误,那你重启一下你的keepalived服务就能看到日志内容了

###################################################################
###################################################################
如果要keepalived配合LVS使用,使用下面的配置文件即可。master即KEEP1,BACKCP即KEEP2.

MASTER:! Configuration File for keepalived

global_defs {
notification_email {
759685538@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_1
}

vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 55
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.100/24
# 10.0.0.200/25
}
}
####################################
virtual_server 10.0.0.100 80{
delay_loop 3
lb_algo rr
lb_kind DR
persistence_timeout 1
protocol TCP
real_server 10.0.0.3 80 {
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 10.0.0.4 80 {
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
###################################################################BACKUP:! Configuration File for keepalived

global_defs {
notification_email {
759685538@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_2
}

vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 55
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.100/24
# 10.0.0.200/25
}
}
################################################
virtual_server 10.0.0.100 80{
delay_loop 3
lb_algo rr
lb_kind DR
persistence_timeout 1
protocol TCP
real_server 10.0.0.3 80 {
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 10.0.0.4 80 {
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
###################################################################

后端的RS服务配置好web服务之后再执行下面命令即可:
每台RS配置VIP
ifconfig lo:100 10.0.0.100 netmask 255.255.255.255 up
ifconfig lo:100
ifconfig
route add -host 10.0.0.100 dev lo
抑制ARP响应
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

或者用下面脚本执行:
cat /etc/init.d/realserver.sh
#!/bin/bash
# description: Config realserver lo and apply noarp

VIP=10.0.0.100

/etc/rc.d/init.d/functions

case "$1" in
start)
ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
/sbin/route add -host $VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"

;;
stop)
ifconfig lo:0 down
route del $VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac

exit 0

###################################################################
###################################################################
问题解决
故障1:
[root@KEEP1 keepalived-1.2.10]# ./configure

checking for sys/time.h... yes
checking openssl/ssl.h usability... no
checking openssl/ssl.h presence... no
checking for openssl/ssl.h... no
configure: error:
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files.
yum install openssl* -y

故障2
[root@KEEP1 keepalived-1.2.10]# ./configure

libipvs.c:73: error: ‘NLE_PERM’ undeclared (first use in this function)
libipvs.c:74: error: ‘NLE_BUSY’ undeclared (first use in this function)
libipvs.c:75: error: ‘NLE_RANGE’ undeclared (first use in this function)
libipvs.c:76: error: ‘NLE_NODEV’ undeclared (first use in this function)
make[2]: *** [libipvs.o] Error 1
make[2]: Leaving directory `/home/lvnian/tools/keepalived-1.2.10/keepalived/libipvs-2.6'
make[1]: *** [all] Error 1
make[1]: Leaving directory `/home/lvnian/tools/keepalived-1.2.10/keepalived'
make: *** [all] Error 2
解决办法
[root@KEEP1 keepalived-1.2.10]# yum install libnl* -y

本文出自 “奋斗吧” 博客,请务必保留此出处http://lvnian.blog.51cto.com/7155281/1702298
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: