corosync+pacemaker配置高可用集群

一、环境介绍：
Node1：
CentOS6 192.168.9.168 192-168-9-168
CentOS6 192.168.9.176 192-168-9-176
VIP： 192.168.9.144
corosync和pacemaker为yum安装
二、环境部署
服务器初始化脚本执行（主要时间同步，防火墙，主机名，ip配置，关闭不必要服务等）
1，更改yum源
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo
yum clean all

2，安装epel
rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm -Uvh http://mirrors.kernel.org/fedora-epel/epel-release-latest-6.noarch.rpm

3，设置主机名
hostname
vim /etc/sysconfig/network

4，绑定host
cat /etc/hosts
192.168.9.168 192-168-9-168
192.168.9.176 192-168-9-176

可配置秘钥通信：
ssh-keygen -P ''
ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.9.168
ssh 192.168.9.168 'date'

5，安装corosync+pacemaker
yum install -y corosync pacemaker

安装httpd
yum install -y httpd

6，配置
# grep -v "^[:space:]\{0,\}.*#" corosync.conf | grep -v ^$
compatibility: whitetank
totem {
version: 2
secauth: on
threads: 0
interface {
ringnumber: 0
bindnetaddr: 192.168.9.0
mcastaddr: 239.255.1.91
mcastport: 5405
ttl: 1
}
}
logging {
fileline: off
to_stderr: no
to_logfile: yes
logfile: /var/log/cluster/corosync.log
to_syslog: no
debug: off
timestamp: on
logger_subsys {
subsys: AMF
debug: off
}
}
service {
ver: 0
name: pacemaker
}
aisexec {
user: root
group: root
}

7，启动
service corosync start

8，查看启动是否正常
crm_mon
tail /var/log/cluster/corosync.log
grep -e "Corosync Cluster Engine" -e "configuration file" /var/log/cluster/corosync.log
grep TOTEM /var/log/cluster/corosync.log
grep ERROR: /var/log/cluster/corosync.log | grep -v unpack_resources
grep pcmk_startup /var/log/cluster/corosync.log

9，crm使用方法
http://blog.51yip.com/server/1680.html
（1）yum安装
cd /etc/yum.repos.d/ ; wget http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/network:ha-clustering:Stable.repo
yum install -y crmsh*
（2）×××下载包
http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/noarch/
yum install -y *.rpm
（3）crm使用
# crm #输入crm命令，进入crm sh 模式
crm(live)# help #输入help查看帮助
crm(live)# configure #输入configure进入configure模式下，
crm(live)configure# #敲两下tab键就会显示configure下全部命令
crm(live)configure# help node
crm(live)configure# cd ../ #返回上一级
1）查看集群状态
crm(live)# status
2）查看一下CIB配置：
crm(live)# configure
crm(live)configure# show
crm(live)configure# show xml #查xml格式文件
3）检测一下配置文件是否有错，可以看到刚开始默认STONITH配置错误，因为没有STONITH设备
crm(live)configure# verify
4）查看当前集群系统所支持的资源代理类型
crm(live)# ra
crm(live)ra# classes
5）查看某种类别下的所用资源代理的列表
crm(live)ra# list lsb
crm(live)ra# list ocf heartbeat
crm(live)ra# list ocf pacemaker
6）查看某个资源代理的配置方法，例如查看了heartbeat的ip资源的配置方法：
crm(live)ra# info ocf:heartbeat:IPaddr
7）节点下线，上线
crm node standby
crm node online
crm status
crm node status
8）资源管理
crm(live)# resource
crm(live)# resource
crm(live)resource# status vip
crm(live)resource# migrate vip 192-168-9-176
crm(live)resource# unmigrate
crm(live)resource# stop webservice
9）排列约束
crm(live)configure# colocation webserver_with_webip inf: webserver webip
10）顺序约束
crm(live)configure# order webip_before_webserver Mandatory: webip webserver
11）位置约束
crm(live)configure# location webip_on_node2 webip 50: 192-168-9-176

10，crmsh配置集群资源
说明：vip，nfs，httpd三个资源配置到资源组，并按顺序启动；双节点投票设置为ignore并添加监控
（1）解决stonith配置错误，设置票数失效操作
crm(live)# configure
crm(live)configure# show
node 192-168-9-168
node 192-168-9-176
property cib-bootstrap-options: \
have-watchdog=false \
dc-version=1.1.15-5.el6-e174ec8 \
cluster-infrastructure="classic openais (with plugin)" \
expected-quorum-votes=2
crm(live)configure# verify
ERROR: error: unpack_resources: Resource start-up disabled since no STONITH resources have been defined
error: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option
error: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity
Errors found during check: config not valid
crm(live)configure# property stonith-enabled=false
crm(live)configure# property no-quorum-policy=ignore
crm(live)configure# verify
crm(live)configure# commit
（2）创建IP地址资源，IP资源是主资源
crm(live)configure# primitive vip ocf:heartbeat:IPaddr params ip=192.168.9.144 nic=eth0 cidr_netmask=24 op monitor interval=10s timeout=20s
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
node 192-168-9-168
node 192-168-9-176
primitive vip IPaddr \
params ip=192.168.9.144 nic=eth0 cidr_netmask=24
property cib-bootstrap-options: \
have-watchdog=false \
dc-version=1.1.15-5.el6-e174ec8 \
cluster-infrastructure="classic openais (with plugin)" \
expected-quorum-votes=2 \
stonith-enabled=false
（3）配置nfs和httpd服务
crm(live)configure# primitive nfs ocf:heartbeat:Filesystem params device=192.168.9.225:/web/hahtml directory=/var/www/html fstype=nfs op monitor interval=20s timeout=40s
crm(live)configure# primitive httpd lsb:httpd op monitor interval=10s timeout=20s
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
node 192-168-9-168
node 192-168-9-176
primitive httpd lsb:httpd
primitive nfs Filesystem \
params device="192.168.9.225:/web/hahtml" directory="/var/www/html" fstype=nfs
primitive vip IPaddr \
params ip=192.168.9.144 nic=eth0 cidr_netmask=24
property cib-bootstrap-options: \
have-watchdog=false \
dc-version=1.1.15-5.el6-e174ec8 \
cluster-infrastructure="classic openais (with plugin)" \
expected-quorum-votes=2 \
stonith-enabled=false
crm(live)configure# cd
crm(live)# status
Stack: classic openais (with plugin)
Current DC: 192-168-9-168 (version 1.1.15-5.el6-e174ec8) - partition with quorum
Last updated: Fri Mar 2 14:53:37 2018 Last change: Fri Mar 2 14:52:46 2018 by root via cibadmin on 192-168-9-168
, 2 expected votes
2 nodes and 3 resources configured

Online: [ 192-168-9-168 192-168-9-176 ]

Full list of resources:

vip (ocf::heartbeat:IPaddr): Started 192-168-9-168
nfs (ocf::heartbeat:Filesystem): Started 192-168-9-176
httpd (lsb:httpd): Started 192-168-9-168
（4）创建资源组，将三个资源在同一节点启动
crm(live)# configure
crm(live)configure# group group_webservice vip nfs httpd
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
node 192-168-9-168
node 192-168-9-176
primitive httpd lsb:httpd
primitive nfs Filesystem \
params device="192.168.9.225:/web/hahtml" directory="/var/www/html" fstype=nfs
primitive vip IPaddr \
params ip=192.168.9.144 nic=eth0 cidr_netmask=24
group group_webservice vip nfs httpd
property cib-bootstrap-options: \
have-watchdog=false \
dc-version=1.1.15-5.el6-e174ec8 \
cluster-infrastructure="classic openais (with plugin)" \
expected-quorum-votes=2 \
stonith-enabled=false
crm(live)configure# cd
crm(live)# status
Stack: classic openais (with plugin)
Current DC: 192-168-9-168 (version 1.1.15-5.el6-e174ec8) - partition with quorum
Last updated: Fri Mar 2 14:55:12 2018 Last change: Fri Mar 2 14:55:04 2018 by root via cibadmin on 192-168-9-168
, 2 expected votes
2 nodes and 3 resources configured

Online: [ 192-168-9-168 192-168-9-176 ]

Full list of resources:

Resource Group: group_webservice
vip (ocf::heartbeat:IPaddr): Started 192-168-9-168
nfs (ocf::heartbeat:Filesystem): Started 192-168-9-168
httpd (lsb:httpd): Started 192-168-9-168

11，测试
（1）使节点为standy 测试切换
（2）service corosync stop关闭corosync 测试选票系统
（3）killall httpd服务测试监控