基于corosync+ldirectord实现director高可用

说明：本篇内容是基于另一篇《LVS负载均衡集群详解》为前提，所以一些基本的上篇文档详细介绍过的内容不再重复。

注意：配置前提
1、设置各个节点间的时间同步
2、基于hosts文件实现能够互相用主机名访问
3、使用uname -n执行结果要和主机名相同
4、确保ipvsadm服务和httpd服务等只要被设置为资源的服务关闭开机启动

一、环境拓扑介绍





与上篇《基于heartbeat v2和ldirectord现实director高可用》相同，能够实现director节点之间的故障转移，node节点的故障剔除，恢复自动加入。

二、安装相关软件包
依赖的软件：
ipvsadm
cluster-glue
cluster-glue-libs
corosync
corosynclib
heartbeat
heartbeat-ldirectord
heartbeat-libs
libesmtp
pacemaker
pacemaker-cts
pacemaker-libs
perl-MailTools
resource-agents
libibverbs
libnes
librdmacm
libtool-ltdl
lm_sensors
openhpi-libs
openib
perl-Compress-Zlib
perl-HTML-Parser
perl-HTML-Tagset
perl-TimeDate
perl-libwww-perl

所有软件包均为rpm包，需要在director1和director2两个节点安装，这里不再给出软件包的具体安装过程。

二、配置corosync整合pacemaker

[root@director1 ~]# chkconfig heartbeat off //关闭heartbeat的开机启动
[root@director1 ~]# cd /etc/corosync/
[root@director1 corosync]# cp corosync.conf.example corosync.conf
[root@director1 corosync]# vim corosync.conf

------------------------文件内容------------------------------------
compatibility: whitetank //打开兼容corosync-0.8之前的版本
totem { //定义节点之间心跳信息如何传递
version: 2 //协议版本
secauth: off //是否开启节点之间安全认证
threads: 0 //启动的线程，与CPU个数相等，0为自动管理
interface { //定义传递心跳的接口信息
ringnumber: 0 //如果有多个接口，ringunmber不能相同
bindnetaddr: 192.168.56.0 //指定接口所在的网络或者接口的IP地址
mcastaddr: 226.94.1.1 //指定多播地址
mcastport: 5405 //多播的端口
}
}
logging { //定义日志相关信息
fileline: off
to_stderr: no //是否把错误信息发送到标准输出
to_logfile: yes //是否存储到logfile中指定的日志文件
to_syslog: yes //是否存储到系统日志文件也就是messages
logfile: /var/log/cluster/corosync.log //日志文件存放路径
debug: off //是否开启调试
timestamp: on //日志信息是否记录时间戳
logger_subsys { //定义日志子系统
subsys: AMF
debug: off
}
}
amf { //定义amf相关信息，如果要启用需安装openais和openais-lib
mode: disabled
}
service { //自定义的服务
ver: 0 //版本
name: pacemaker //整合pacemaker，当corosync启动时也启动pacemaker
}
--------------------------------内容结束-----------------------------------
[root@director1 corosync]# scp corosync.conf director2:/etc/corosync/ //把配置文件拷贝到director2节点
[root@director1 corosync]# mkdir /var/log/cluster //创建日志存放目录
[root@director1 corosync]# ssh director2 'mkdir /var/log/cluster' //在director2节点上创建日志存放目录

二、启动corosync服务

[root@director1 corosync]# service corosync start //启动director1上的corosync服务
[root@director1 corosync]# netstat -unlp | grep 5404 //查看是否有corosync进程监听udp的5404端口

在director2节点上启动corosync服务
[root@director1 corosync]# ssh director2 'service corosync start'

查看两个节点是否正常
[root@director1 corosync]# crm status
Online: [ director1 director2 ] //看到这样的行说明两个节点已经在线

三、配置ldirectord配置文件
[root@director1 ~]# cp /usr/share/doc/heartbeat-ldirectord-2.1.4/ldirectord.cf /etc/ha.d/ //拷贝配置文件样例
[root@director1 ~]# vim /etc/ha.d/ldirectord.cf
---------------------------文件内容-------------------------------
checktimeout=3
checkinterval=1
autoreload=yes
quiescent=yes
virtual=192.168.56.200:80
real=192.168.56.103:80 gate
real=192.168.56.104:80 gate
fallback=127.0.0.1:80 gate
service=http
request=".test.html"
receive="ok"
scheduler=wlc
protocol=tcp
checktype=negotiate
checkport=80
--------------------------------结束-------------------------------------
配置文件内容的具体说明已在上一篇文档《基于heartbeat v2和ldirectord现实director高可用》中详细说明了，这里不再重复描述
拷贝ldirectord配置文件到director2节点
[root@director1 ~]# scp /etc/ha.d/ldirectord.cf director2:/etc/ha.d/

三、配置资源

1、关闭STONITH检测，由于我这里没有STONITH设置所以关闭此项，而且因为此环境中只有两个节点所以应该关闭票数策略。
[root@director1 ~]# crm
crm(live)# configure
crm(live)configure# property stonith-enabled=false
crm(live)configure# property no-quorum-policy=ignore
crm(live)configure# commit
crm(live)configure# exit

2、配置资源

[root@director1 ~]# crm
crm(live)# configure
crm(live)configure# primitive VIP ocf:heartbeat:IPaddr params ip=192.168.56.200 nic=eth0:0 cidr_netmask=255.255.255.255 broadcast=192.168.56.200 //配置VIP资源

crm(live)configure# primitive LVS lsb:ldirectord //配置ldirectord资源

crm(live)configure# colocation VIP_with_LVS inf: VIP LVS //定义排列约束，使VIP和LVS两个资源必须同时在一个节点上

crm(live)configure# location conn1 VIP 100: director2 //定义位置约束，使VIP资源更倾向于运行与director2节点

crm(live)configure# commit //提交配置
crm(live)configure# exit //退出

四、测试

1、在director2节点上验证
[root@director2 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 08:00:27:DB:A2:9B
inet addr:192.168.56.102 Bcast:192.168.56.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:41132 errors:0 dropped:0 overruns:0 frame:0
TX packets:29820 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8197161 (7.8 MiB) TX bytes:3815265 (3.6 MiB)

eth0:0 Link encap:Ethernet HWaddr 08:00:27:DB:A2:9B
inet addr:192.168.56.200 Bcast:192.168.56.200 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4338 errors:0 dropped:0 overruns:0 frame:0
TX packets:4338 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:426216 (416.2 KiB) TX bytes:426216 (416.2 KiB)

[root@director2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.56.200:80 wlc
-> 192.168.56.103:80 Route 0 0 0
-> 192.168.56.104:80 Route 0 0 0
-> 127.0.0.1:80 Local 1 0 0
可以看到定义的两个资源已经在director2节点上生效

2、模拟director2节点故障，测试能否把资源转移到director1节点
[root@director2 ~]# crm node standby //使当前节点转为备用节点

在director1节点上验证：
[root@director1 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 08:00:27:EF:F7:44
inet addr:192.168.56.101 Bcast:192.168.56.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:19310 errors:0 dropped:0 overruns:0 frame:0
TX packets:24406 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2381082 (2.2 MiB) TX bytes:2949759 (2.8 MiB)

eth0:0 Link encap:Ethernet HWaddr 08:00:27:EF:F7:44
inet addr:192.168.56.200 Bcast:192.168.56.200 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:160 errors:0 dropped:0 overruns:0 frame:0
TX packets:160 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:13936 (13.6 KiB) TX bytes:13936 (13.6 KiB)

[root@director1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.56.200:80 wlc
-> 192.168.56.103:80 Route 0 0 0
-> 192.168.56.104:80 Route 0 0 0
-> 127.0.0.1:80 Local 1 0 0
可以看到资源已经成功转移到director1节点

3、重新让director2上线验证定义的位置约束是生效
[root@director2 ~]# crm node online
[root@director2 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 08:00:27:DB:A2:9B
inet addr:192.168.56.102 Bcast:192.168.56.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:47245 errors:0 dropped:0 overruns:0 frame:0
TX packets:33846 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8876133 (8.4 MiB) TX bytes:4296712 (4.0 MiB)

eth0:0 Link encap:Ethernet HWaddr 08:00:27:DB:A2:9B
inet addr:192.168.56.200 Bcast:192.168.56.200 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4377 errors:0 dropped:0 overruns:0 frame:0
TX packets:4377 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:429648 (419.5 KiB) TX bytes:429648 (419.5 KiB)

[root@director2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.56.200:80 wlc
-> 192.168.56.103:80 Route 0 0 0
-> 192.168.56.104:80 Route 0 0 0
-> 127.0.0.1:80 Local 1 0 0
验证成功。本文出自 “夜，幕” 博客，请务必保留此出处http://yooke.blog.51cto.com/4472498/957468