通过Keepalived实现Redis Failover自动故障切换功能
2012-12-10 20:01
816 查看
通过Keepalived实现Redis Failover自动故障切换功能[实践分享]
目前,Redis还没有一个类似于MySQL Proxy或Oracle RAC的官方HA方案。Redis作者有一个名为Redis Sentinel的计划(http://redis.io/topics/sentinel),据称将会有监控,报警和自动故障转移三大功能,非常不错。
但可惜的是短期内恐怕还不能开发完成。
因此,如何在出现故障时自动转移是一个需要解决的问题。
通过对网上一些资料的搜索,有建议采用HAProxy或Keepalived来实现的,事实上如果是做Failover而非负载均衡的话,Keepalived的效率肯定是超过HAProxy的,所以我决定采用Keepalived的方案。
环境介绍:
Master: 10.6.1.143
Slave: 10.6.1.144
Virtural IP Address (VIP): 10.6.1.200
设计思路:
当 Master 与 Slave 均运作正常时, Master负责服务,Slave负责Standby;
当 Master 挂掉,Slave 正常时, Slave接管服务,同时关闭主从复制功能;
当 Master 恢复正常,则从Slave同步数据,同步数据之后关闭主从复制功能,恢复Master身份,于此同时Slave等待Master同步数据完成之后,恢复Slave身份。
然后依次循环。
需要注意的是,这样做需要在Master与Slave上都开启本地化策略,否则在互相自动切换的过程中,未开启本地化的一方会将另一方的数据清空,造成数据完全丢失。
下面,是具体的实施步骤:
在Master和Slave上安装Keepalived
$ sudo apt-get install keepalived
修改Master和Slave的/etc/hosts文件
$ sudo vim /etc/hosts
1 | 127.0.0.1 localhost |
2 | 10.6.1.143 redis |
3 | 10.6.1.144 redis-slave |
首先,在Master上创建如下配置文件:
$ sudo vim /etc/keepalived/keepalived.conf
01 | vrrp_script chk_redis { |
02 | script "/etc/keepalived/scripts/redis_check.sh" ###监控脚本 |
03 | interval ###监控时间 |
04 | } |
05 | vrrp_instance VI_1 { |
06 | state ###设置为MASTER |
07 | interface ###监控网卡 |
08 | virtual_router_id 51 |
09 | priority ###权重值 |
10 | authentication { |
11 | auth_type PASS ###加密 |
12 | auth_pass ###密码 |
13 | } |
14 | track_script { |
15 | chk_redis ###执行上面定义的chk_redis |
16 | } |
17 | virtual_ipaddress { |
18 | 10.6.1.200 ###VIP |
19 | } |
20 | notify_master /etc/keepalived/scripts/redis_master.sh |
21 | notify_backup /etc/keepalived/scripts/redis_backup.sh |
22 | notify_fault /etc/keepalived/scripts/redis_fault.sh |
23 | notify_stop /etc/keepalived/scripts/redis_stop.sh |
24 | } |
$ sudo vim /etc/keepalived/keepalived.conf
01 | vrrp_script chk_redis { |
02 | script "/etc/keepalived/scripts/redis_check.sh" ###监控脚本 |
03 | interval ###监控时间 |
04 | } |
05 | vrrp_instance VI_1 { |
06 | state ###设置为BACKUP |
07 | interface ###监控网卡 |
08 | virtual_router_id 51 |
09 | priority ###比MASTRE权重值低 |
10 | authentication { |
11 | auth_type PASS |
12 | auth_pass ###密码与MASTRE相同 |
13 | } |
14 | track_script { |
15 | chk_redis ###执行上面定义的chk_redis |
16 | } |
17 | virtual_ipaddress { |
18 | 10.6.1.200 ###VIP |
19 | } |
20 | notify_master /etc/keepalived/scripts/redis_master.sh |
21 | notify_backup /etc/keepalived/scripts/redis_backup.sh |
22 | notify_fault /etc/keepalived/scripts/redis_fault.sh |
23 | notify_stop /etc/keepalived/scripts/redis_stop.sh |
24 | } |
$ sudo mkdir /etc/keepalived/scripts
$ sudo vim /etc/keepalived/scripts/redis_check.sh
01 | #!/bin/bash |
02 |
03 | ALIVE=`/opt/redis/bin/redis-cli PING` |
04 | if [ "$ALIVE" == "PONG" ]; then |
05 | echo $ALIVE |
06 | exit 0 |
07 | else |
08 | echo $ALIVE |
09 | exit 1 |
10 | fi |
notify_master /etc/keepalived/scripts/redis_master.sh
notify_backup /etc/keepalived/scripts/redis_backup.sh
notify_fault /etc/keepalived/scripts/redis_fault.sh
notify_stop /etc/keepalived/scripts/redis_stop.sh
因为Keepalived在转换状态时会依照状态来呼叫:
当进入Master状态时会呼叫notify_master
当进入Backup状态时会呼叫notify_backup
当发现异常情况时进入Fault状态呼叫notify_fault
当Keepalived程序终止时则呼叫notify_stop
首先,在Redis Master上创建notity_master与notify_backup脚本:
$ sudo vim /etc/keepalived/scripts/redis_master.sh
01 | #!/bin/bash |
02 |
03 | REDISCLI= "/opt/redis/bin/redis-cli" |
04 | LOGFILE= "/var/log/keepalived-redis-state.log" |
05 |
06 | echo "[master]" >> $LOGFILE |
07 | date >> $LOGFILE |
08 | echo "Being master...." >> $LOGFILE 2>&1 |
09 |
10 | echo "Run SLAVEOF cmd ..." >> $LOGFILE |
11 | $REDISCLI |
12 | sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态 |
13 |
14 | echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE |
15 | $REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1 |
01 | #!/bin/bash |
02 |
03 | REDISCLI= "/opt/redis/bin/redis-cli" |
04 | LOGFILE= "/var/log/keepalived-redis-state.log" |
05 |
06 | echo "[backup]" >> $LOGFILE |
07 | date >> $LOGFILE |
08 | echo "Being slave...." >> $LOGFILE 2>&1 |
09 |
10 | sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色 |
11 | echo "Run SLAVEOF cmd ..." >> $LOGFILE |
12 | $REDISCLI |
$ sudo vim /etc/keepalived/scripts/redis_master.sh
01 | #!/bin/bash |
02 |
03 | REDISCLI= "/opt/redis/bin/redis-cli" |
04 | LOGFILE= "/var/log/keepalived-redis-state.log" |
05 |
06 | echo "[master]" >> $LOGFILE |
07 | date >> $LOGFILE |
08 | echo "Being master...." >> $LOGFILE 2>&1 |
09 |
10 | echo "Run SLAVEOF cmd ..." >> $LOGFILE |
11 | $REDISCLI |
12 | sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态 |
13 |
14 | echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE |
15 | $REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1 |
01 | #!/bin/bash |
02 |
03 | REDISCLI= "/opt/redis/bin/redis-cli" |
04 | LOGFILE= "/var/log/keepalived-redis-state.log" |
05 |
06 | echo "[backup]" >> $LOGFILE |
07 | date >> $LOGFILE |
08 | echo "Being slave...." >> $LOGFILE 2>&1 |
09 |
10 | sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色 |
11 | echo "Run SLAVEOF cmd ..." >> $LOGFILE |
12 | $REDISCLI |
$ sudo vim /etc/keepalived/scripts/redis_fault.sh
1 | #!/bin/bash |
2 |
3 | LOGFILE=/var/log/keepalived-redis-state.log |
4 |
5 | echo "[fault]" >> $LOGFILE |
6 | date >> $LOGFILE |
1 | #!/bin/bash |
2 |
3 | LOGFILE=/var/log/keepalived-redis-state.log |
4 |
5 | echo "[stop]" >> $LOGFILE |
6 | date >> $LOGFILE |
$ sudo chmod +x /etc/keepalived/scripts/*.sh
脚本创建完成以后,我们开始按照如下流程进行测试:
1.启动Master上的Redis
$ sudo /etc/init.d/redis start
2.启动Slave上的Redis
$ sudo /etc/init.d/redis start
3.启动Master上的Keepalived
$ sudo /etc/init.d/keepalived start
4.启动Slave上的Keepalived
$ sudo /etc/init.d/keepalived start
5.尝试通过VIP连接Redis:
$ redis-cli -h 10.6.1.200 INFO
连接成功,Slave也连接上来了。
role:master
slave0:10.6.1.144,6379,online
6.尝试插入一些数据:
$ redis-cli -h 10.6.1.200 SET Hello Redis
OK
从VIP读取数据
$ redis-cli -h 10.6.1.200 GET Hello
"Redis"
从Master读取数据
$ redis-cli -h 10.6.1.143 GET Hello
"Redis"
从Slave读取数据
$ redis-cli -h 10.6.1.144 GET Hello
"Redis"
下面,模拟故障产生:
将Master上的Redis进程杀死:
$ sudo killall -9 redis-server
查看Master上的Keepalived日志
$ tailf /var/log/keepalived-redis-state.log
[fault]
Thu Sep 27 08:29:01 CST 2012
同时Slave上的日志显示:
$ tailf /var/log/keepalived-redis-state.log
[master]
Fri Sep 28 14:14:09 CST 2012
Being master....
Run SLAVEOF cmd ...
OK
Run SLAVEOF NO ONE cmd ...
OK
然后我们可以发现,Slave已经接管服务,并且担任Master的角色了。
$ redis-cli -h 10.6.1.200 INFO
$ redis-cli -h 10.6.1.144 INFO
role:master
然后我们恢复Master的Redis进程
$ sudo /etc/init.d/redis start
查看Master上的Keepalived日志
$ tailf /var/log/keepalived-redis-state.log
[master]
Thu Sep 27 08:31:33 CST 2012
Being master....
Run SLAVEOF cmd ...
OK
Run SLAVEOF NO ONE cmd ...
OK
同时Slave上的日志显示:
$ tailf /var/log/keepalived-redis-state.log
[backup]
Fri Sep 28 14:16:37 CST 2012
Being slave....
Run SLAVEOF cmd ...
OK
可以发现目前的Master已经再次恢复了Master的角色,故障切换以及自动恢复都成功了。
Failover, Keepalived, Redis
版权所有© mcsrainbow,保留所有原创日志的权利。转载请注明出处:http://heyLinux.com 。
这篇文章发表于 2012/09/28 14:33,属于 Linux&Unix 分类。 你可以通过 RSS
2.0 来跟踪这篇文章。 你还可以对它进行 评论。
Comments
(6)
Related
Posts
#1 by 过客 on
2012/10/30 - 11:08
你好,我想问一下,那个VIP的IP是怎么来的?
#2 by tnt on
2012/11/16 - 10:20
请好,我按照您的方法做很顺利 但是当到最后的时候:
我的主服务器也是如下的日志:
$ tailf /var/log/keepalived-redis-state.log
[fault]
Thu Sep 27 08:29:01 CST 2012
但是从服务器的日志如下:
[master]
Fri Nov 16 10:12:15CST 2012
Being master....
Run SLAVEOF cmd ...
Could not connect to Redis at 127.0.0.1:6379: Connection refused
Run SLAVEOF NO ONE cmd ...
Could not connect to Redis at 127.0.0.1:6379: Connection refused
请问这个为什么呢 麻烦大神了 谢谢
#3 by mcsrainbow on
2012/11/16 - 11:43
Redis连接不上,检查下Redis的配置,关于Redis的安装配置我博客上也有。
#4 by tnt on
2012/11/16 - 14:13
重新调整过后现在是
当master端 redis 服务停掉后 VIP迁移不到 slave端;并且也不出现在master端了
当master端 redis服务起来后 VIP也是不出现的 要重启keepalived才能再次出现VIP 请问是什么问题类
#5 by tnt on
2012/11/16 - 14:49
LZ 我解决问题 问题是在同一网段内virtual_router_id 值不能相同,如果相同会在messages中收到VRRP错误包
所以在master virtual_router_id 51改成50就可以了
谢谢 LZ分享
#6 by tnt on
2012/11/16 - 17:18
悲剧,当master端 redis 服务停掉后 VIP迁移不到 slave端;并且也不出现在master端了 又出现这种现象了 在从服务器上面 重复出现这种日志
Nov 16 16:16:03 localhost Keepalived_vrrp: ip address associated with VRID not present
in received packet : -1358845760
Nov 16 16:16:03 localhost Keepalived_vrrp: one or more VIP associated with VRID mismatc
h actual MASTER advert
Nov 16 16:16:03 localhost Keepalived_vrrp: bogus VRRP packet received on eth0 !!!
Nov 16 16:16:03 localhost Keepalived_vrrp: VRRP_Instance(VI_1) ignoring received advert
isment...
LZ 求助啊
相关文章推荐
- 通过Keepalived实现Redis Failover自动故障切换功能(整理中) .
- 通过Keepalived实现Redis Failover自动故障切换功能[实践分享]
- 通过Keepalived实现Redis Failover自动故障切换功能
- 通过Keepalived实现Redis Failover自动故障切换功能[实践分享]
- 通过Keepalived实现Redis Failover自动故障切换功能
- 通过Keepalived实现Redis Failover自动故障切换功能
- 通过Keepalived实现Redis Failover自动故障切换功能
- 通过Keepalived实现Redis Failover自动故障切换功能
- 通过Keepalived实现Redis Failover自动故障切换功能
- Redis主从复制 - 通过Keepalived实现Redis Failover自动故障切换功能
- 通过Keepalived实现Redis Failover自动故障切换功能[实践分享] =转载
- 通过Keepalived实现Redis Failover自动故障切换功能[实践分享] =转载
- 通过Keepalived实现Redis Failover自动故障切换
- Keepalived实现Redis Failover自动故障切换
- 结合keepalived实现redis群集高可用故障自动切换 推荐
- redis主从配置及通过keepalived实现redis自动切换,redis主从实现10秒检查与恢复
- keepalived+redis 实现高可用的自动故障转移failover
- Redis主从配置及通过Keepalived实现Redis自动切换高可用
- 结合keepalived实现redis群集高可用故障自动切换
- Redis主从配置及通过Keepalived实现Redis自动切换高可用