Redis + Keepalived主从集群的搭建及故障转移

Redis + Keepalived主从集群的搭建及故障转移

设计思路：

当 Master 与 Slave 均运作正常时, Master负责服务，Slave负责Standby；

当 Master 挂掉，Slave 正常时, Slave接管服务，同时关闭主从复制功能；

当 Master 恢复正常，则从Slave同步数据，同步数据之后关闭主从复制功能，恢复Master身份，于此同时Slave等待Master同步数据完成之后，恢复Slave身份。

然后依次循环。

注意事项:

需要在Master与Slave上都开启本地化策略，否则在互相自动切换的过程中，未开启本地化的一方会将另一方的数据清空，造成数据完全丢失。

环境准备

集群可以是单台机器的不同端口，也可以是多台机器，通常实际应用是，多台机器。假设目前已经安装好了2台机器，IP和端口分别是

主Redis服务器IP：192.168.1.148
主Redis服务器端口号：6379
从Redis服务器IP：192.168.1.158
从Redis服务器端口号：6379

设定一个虚拟IP

192.168.1.200

进入root用户

xiaoyao@xiaoyao-virtual-machine:~$ su
密码：
root@xiaoyao-virtual-machine:/home/xiaoyao#

主从两个机器上分别安装Keepalived

root@xiaoyao-virtual-machine:/home/xiaoyao# apt-get install keepalived

主从服务器分别修改hosts文件

root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/hosts
在文件末尾添加
192.168.1.148 redis148
192.168.1.158 redis158

主服务器创建配置文件

root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/keepalived.conf

配置内容：

! Configuration File for keepalived
global_defs {
router_id redis148
}
vrrp_script chk_redis
{
script "/etc/keepalived/scripts/redis_check.sh 127.0.0.1 6379"
interval 2
timeout 2
fall 3
}
vrrp_instance redis {
state MASTER # master set to SLAVE also
interface eth0
virtual_router_id 50
priority  150
nopreempt # no seize,must add
advert_int 1
authentication {   #all node must same
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.200/24
}
track_script {
chk_redis
}
notify_master "/etc/keepalived/scripts/redis_master.sh 127.0.0.1 192.168.1.158 6379"
notify_backup "/etc/keepalived/scripts/redis_backup.sh 127.0.0.1 192.168.1.158 6379"
notify_fault /etc/keepalived/scripts/redis_fault.sh
notify_stop /etc/keepalived/scripts/redis_stop.sh
}

这里可能会出错，一般虚拟机配置成：interface eth0，但是有的机器可能是em1，这个要根据情况来配置，否则创建虚拟IP会失败。

举个例子：

查看IP后是下面的样子

root@ubuntu:/etc/init.d# cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto em1
iface em1 inet static
address 192.168.0.33
netmask 255.255.255.0
gateway 192.168.0.1

那么就应该配置成

interface em1

从服务器创建配置文件

root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/keepalived.conf

配置内容：

! Configuration File for keepalived

global_defs {
router_id redis158
}
vrrp_script chk_redis
{
script "/etc/keepalived/scripts/redis_check.sh 127.0.0.1 6379"
interval 2
timeout 2
fall 3
}

vrrp_instance redis {
state BACKUP
interface eth0
virtual_router_id 50
priority  100
advert_int 1
authentication {   #all node must same
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.200/24
}
track_script {
chk_redis
}
notify_master "/etc/keepalived/scripts/redis_master.sh 127.0.0.1 192.168.1.148 6379"
notify_backup "/etc/keepalived/scripts/redis_backup.sh 127.0.0.1 192.168.1.148 6379"
notify_fault /etc/keepalived/scripts/redis_fault.sh
notify_stop /etc/keepalived/scripts/redis_stop.sh
}

在主服务器和从服务器上分别创建监控Redis的脚本

root@xiaoyao-virtual-machine:/home/xiaoyao# mkdir /etc/keepalived/scripts
root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_check.sh

脚本内容：

#!/bin/bash
ALIVE=`/usr/local/bin/redis-cli -h $1 -p $2 PING`
LOGFILE="/var/log/keepalived-redis-check.log"
echo "[CHECK]" >> $LOGFILE
date >> $LOGFILE
if [ $ALIVE == "PONG" ]; then :
echo "Success: redis-cli -h $1 -p $2 PING $ALIVE" >> $LOGFILE 2>&1
exit 0
else
echo "Failed:redis-cli -h $1 -p $2 PING $ALIVE " >> $LOGFILE 2>&1
exit 1
fi

Keepalived在转换状态时会根据状态来呼叫：

当进入Master状态时会呼叫redis_master

当进入Backup状态时会呼叫redis_backup

当发现异常情况时进入Fault状态呼叫redis_fault

当Keepalived程序终止时则呼叫redis_stop

下面在主从服务器分别创建下列脚本

在主服务器上创建redis_master脚本

root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_master.sh

脚本内容

#!/bin/bash
REDISCLI="/usr/local/bin/redis-cli -h $1 -p $3"
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[master]" >> $LOGFILE
date >> $LOGFILE
echo "Being master...." >> $LOGFILE 2>&1
echo "Run MASTER cmd ..." >> $LOGFILE 2>&1
$REDISCLI SLAVEOF $2 $3 >> $LOGFILE
sleep 10 #delay 10 s wait data async cancel sync
echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1

在主服务器上创建redis_backup脚本

root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_backup.sh

脚本内容

#!/bin/bash
REDISCLI="/usr/local/bin/redis-cli"
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[backup]" >> $LOGFILE
date >> $LOGFILE
echo "Run SLAVEOF cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF $2 $3 >> $LOGFILE 2>&1
# echo "Being slave...." >> $LOGFILE 2>&1
sleep 15 #delay 15 s wait data sync exchange role

在主服务器上创建redis_fault脚本

root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_fault.sh

脚本内容

#!/bin/bash
LOGFILE=/var/log/keepaliv
4000
ed-redis-state.log
echo "[fault]" >> $LOGFILE
date >> $LOGFILE

在主服务器上创建redis_stop脚本

root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_stop.sh

脚本内容

#!/bin/bash
LOGFILE=/var/log/keepalived-redis-state.log
echo "[stop]" >> $LOGFILE
date >> $LOGFILE

在从服务器上创建redis_master脚本

root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_master.sh

脚本内容

#!/bin/bash
REDISCLI="/usr/local/bin/redis-cli -h $1 -p $3"
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[master]" >> $LOGFILE
date >> $LOGFILE
echo "Being master...." >> $LOGFILE 2>&1
echo "Run SLAVEOF cmd ... " >> $LOGFILE
$REDISCLI SLAVEOF $2 $3 >> $LOGFILE  2>&1
#echo "SLAVEOF $2 cmd can't excute ... " >> $LOGFILE
sleep 10 ##delay 15 s wait data sync exchange role
echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1

在从服务器上创建redis_backup脚本

root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_backup.sh

脚本内容

#!/bin/bash
REDISCLI="/usr/local/bin/redis-cli"
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[BACKUP]" >> $LOGFILE
date >> $LOGFILE
echo "Being slave...." >> $LOGFILE 2>&1
echo "Run SLAVEOF cmd ..." >> $LOGFILE 2>&1
$REDISCLI SLAVEOF $2 $3 >> $LOGFILE
sleep 100 #delay 10 s wait data async cancel sync
exit(0)

在从服务器上创建redis_fault脚本

root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_fault.sh

脚本内容

#!/bin/bash
LOGFILE=/var/log/keepalived-redis-state.log
echo "[fault]" >> $LOGFILE
date >> $LOGFILE

在从服务器上创建redis_stop脚本

root@xiaoyao-virtual-machine:/home/xiaoyao# vim /etc/keepalived/scripts/redis_stop.sh

脚本内容

#!/bin/bash
LOGFILE=/var/log/keepalived-redis-state.log
echo "[stop]" >> $LOGFILE
date >> $LOGFILE

主服务器和从服务器分别给脚本添加可执行权限

root@xiaoyao-virtual-machine:/home/xiaoyao# chmod +x /etc/keepalived/scripts/*.sh

启动服务测试

启动主服务器上的Redis

root@xiaoyao-virtual-machine:/home/xiaoyao# service redis start

启动从服务器上的Redis

root@xiaoyao-virtual-machine:/home/xiaoyao# service redis start

启动主服务器上的Keepalived

root@xiaoyao-virtual-machine:/home/xiaoyao# /etc/init.d/keepalived start

启动从服务器上的Keepalived

root@xiaoyao-virtual-machine:/home/xiaoyao# /etc/init.d/keepalived start

通过虚拟ip连接Redis

root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.200 INFO
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.1.158,port=6379,state=online,offset=15,lag=1
master_repl_offset:15
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:14

通过虚拟ip插入一条数据

root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.200 SET Hello Redis

主从服务器均能读到此条数据

root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.200 GET Hello
root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.148 GET Hello
root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.158 GET Hello

在主服务器上杀死redis进程

root@xiaoyao-virtual-machine:/home/xiaoyao# killall -9 redis-server

查看主服务器上的Keepalived日志

root@xiaoyao-virtual-machine:/home/xiaoyao# tailf /var/log/keepalived-redis-state.log

查看从服务器上的Keepalived日志

root@xiaoyao-virtual-machine:/home/xiaoyao# tailf /var/log/keepalived-redis-state.log

查看从服务器上的信息

root@xiaoyao-virtual-machine:/home/xiaoyao# redis-cli -h 192.168.1.158 INFO
# Replication
role:master
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:0