利用corosync+pacemaker实现www高可用性群集

-------------利用corosync+pacemaker实现www的高可用性群集--------------------

【高可用群集：
high availability cluster-----简称HA cluster，高可用性群集为了是群集的整体服务尽可能可用，从而减少计算机和软件易错性带来的损失。在群集中如果有一个节点失效或者损坏，它的备用节点就会在很快的时间内接管它的职责，对于使用的用户而言，群集是永远不会停机。又称有两个节点的高可用群集称为双机热备，即两台服务器间相互的备份，高可用群集系统可以支持两个以上的节点； 】

本实验大致的拓扑： 使用到的系统是Linux Redhat 5.4 Enterprise





(1)、修改网络参数：

node1

[root@web1 ~]# vim /etc/sysconfig/network

NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=node1.hanyu.com

[root@web1 ~]# vim /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
192.168.1.101 node1.hanyu.com node1
192.168.1.102 node2.hanyu.com node2

[root@web1 ~]# hostname node1.hanyu.com
[root@web1 ~]# hostname
node1.hanyu.com

node2
root@web2 ~]# vim /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=node2.hanyu.com

[root@web2 ~]# vim /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
192.168.1.101 node1.hanyu.com node1
192.168.1.102 node2.hanyu.com node2

[root@web2 ~]# hostname node2.hanyu.com
[root@web2 ~]# hostname
node2.hanyu.com

(2)、同步系统时间：
[root@web1 ~]# hwclock –s
[root@web2 ~]# hwclock –s

(3)、在各节点上生成密钥实现无密码通信：

node1：
[root@node1 ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
a5:e0:b7:03:7b:95:c1:00:af:00:2a:bc:8e:b9:c1:14 root@node1.hanyu.com 产生一个rsa的非对称加密的私钥对；

[root@node1 ~]# ssh-copy-id -i .ssh/id_rsa.pub node2
15
The authenticity of host 'node2 (192.168.1.102)' can't be established.
RSA key fingerprint is d4:f1:06:3b:a0:81:fd:85:65:20:9e:a1:ee:46:a6:8b.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node2,192.168.1.102' (RSA) to the list of known hosts.
root@node2's password:
Now try logging into the machine, with "ssh 'node2'", and check in:

.ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

[root@node2 ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
47:e6:36:40:c8:72:ba:03:85:87:d2:c3:7a:04:29:c4 root@node2.hanyu.com

[root@node2 ~]# ssh-copy-id -i .ssh/id_rsa.pub node1
15
ssh: node1: Temporary failure in name resolution
[root@node2 ~]# ssh-copy-id -i .ssh/id_rsa.pub 192.168.1.101
15
The authenticity of host '192.168.1.101 (192.168.1.101)' can't be established.
RSA key fingerprint is 78:55:3c:c8:4d:c3:73:91:dc:ae:73:c7:ab:a0:c6:d4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.101' (RSA) to the list of known hosts.
root@192.168.1.101's password:
Now try logging into the machine, with "ssh '192.168.1.101'", and check in:

.ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

(4)、在各节点上配置相应的yum客户端：

在node1和node2的下面配置yum文件中添加以下的内容：
[root@node1 ~]# vim /etc/yum.repos.d/rhel-debuginfo.repo
[root@node2 ~]# vim /etc/yum.repos.d/rhel-debuginfo.repo
[rhel-server]
name=Red Hat Enterprise Linux $releasever - $basearch - Debug
baseurl=file:///mnt/cdrom/Server
enabled=1
gpgcheck=1
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release

[rhel-Cluster]
name=Red Hat Enterprise Linux $releasever - $basearch - Debug
baseurl=file:///mnt/cdrom/Cluster
enabled=1
gpgcheck=1
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release

[rhel-ClusterStorage]
name=Red Hat Enterprise Linux $releasever - $basearch - Debug
baseurl=file:///mnt/cdrom/ClusterStorage
enabled=1
gpgcheck=1
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release

[rhel-VT]
name=Red Hat Enterprise Linux $releasever - $basearch - Debug
baseurl=file:///mnt/cdrom/VT
enabled=1
gpgcheck=1
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release

(5)、本实验需要以下的数据包：

cluster-glue-1.0.6-1.6.el5.i386.rpm
cluster-glue-libs-1.0.6-1.6.el5.i386.rpm
corosync-1.2.7-1.1.el5.i386.rpm
corosynclib-1.2.7-1.1.el5.i386.rpm
drbd83-8.3.8-1.el5.centos.i386.rpm
heartbeat-3.0.3-2.3.el5.i386.rpm
heartbeat-libs-3.0.3-2.3.el5.i386.rpm
kmod-drbd83-8.3.8-1.el5.centos.i686.rpm
ldirectord-1.0.1-1.el5.i386.rpm 探测服务器不需要安装！！！！！！！
libesmtp-1.0.4-5.el5.i386.rpm
mysql-5.5.15-linux2.6-i686.tar.gz
openais-1.1.3-1.6.el5.i386.rpm
openaislib-1.1.3-1.6.el5.i386.rpm
pacemaker-1.1.5-1.1.el5.i386.rpm
pacemaker-cts-1.1.5-1.1.el5.i386.rpm
pacemaker-libs-1.1.5-1.1.el5.i386.rpm
perl-TimeDate-1.16-5.el5.noarch.rpm
resource-agents-1.0.4-1.1.el5.i386.rpm

(6)、在各个节点上安装所有的rpm包：

[root@node1 ~]# yum localinstall *.rpm -y --nogpgcheck
[root@node2 ~]# yum localinstall *.rpm -y –nogpgcheck

(7)、对各个节点进行相应的配置：

node1：
[root@node1 ~]# cd /etc/corosync/
[root@node1 corosync]# cp corosync.conf.example corosync.conf
[root@node1 corosync]# vim corosync.conf
# Please read the corosync.conf.5 manual page
compatibility: whitetank

totem {
version: 2
secauth: off
threads: 0
interface {
ringnumber: 0
bindnetaddr: 192.168.1.0
mcastaddr: 226.94.1.1
mcastport: 5405
}
}
logging {
fileline: off
to_stderr: no
to_logfile: yes
to_syslog: yes
logfile: /var/log/cluster/corosync.log
debug: off
timestamp: on
logger_subsys {
subsys: AMF
debug: off
}
}

amf {
mode: disabled
}

service {
ver: 0
name: pacemaker
}
aisexec {
user: root
group: root
}

[root@node1 corosync]# mkdir /var/log/cluster
[root@node1 corosync]# corosync-keygen
Corosync Cluster Engine Authentication key generator.
Gathering 1024 bits for key from /dev/random.
Press keys on your keyboard to generate entropy.
Writing corosync key to /etc/corosync/authkey.
[root@node1 corosync]# ll
总计 52
-rw-r--r-- 1 root root 5384 2010-07-28 amf.conf.example
-r-------- 1 root root 128 06-04 10:41 authkey
-rw-r--r-- 1 root root 546 06-04 10:35 corosync.conf
-rw-r--r-- 1 root root 436 2010-07-28 corosync.conf.example
drwxr-xr-x 2 root root 4096 2010-07-28 service.d
drwxr-xr-x 2 root root 4096 2010-07-28 uidgid.d

将node1节点上的文件拷贝到节点node2上面 一定要带上 -p
[root@node1 corosync]# scp -p authkey corosync.conf node2:/etc/corosync/
authkey 100% 128 0.1KB/s 00:00
corosync.conf 100% 546 0.5KB/s 00:00

开启node1节点上的corosync服务：
[root@node1 corosync]# service corosync start
Starting Corosync Cluster Engine (corosync): [确定]

验证corosync引擎是否正常工作启动：
[root@node1 corosync]# grep -i -e "corosync cluster engine" -e "configuration file" /var/log/messages
Mar 20 23:06:16 localhost smartd[3286]: Opened configuration file /etc/smartd.conf
Mar 20 23:06:16 localhost smartd[3286]: Configuration file /etc/smartd.conf was parsed, found DEVICESCAN, scanning devices
Jun 4 10:42:30 localhost corosync[23702]: [MAIN ] Corosync Cluster Engine ('1.2.7'): started and ready to provide service.
Jun 4 10:42:30 localhost corosync[23702]: [MAIN ] Successfully read main configuration file '/etc/corosync/corosync.conf'.
Jun 4 10:46:36 localhost corosync[23702]: [MAIN ] Corosync Cluster Engine exiting with status 0 at main.c:170.
Jun 4 10:49:07 localhost corosync[23763]: [MAIN ] Corosync Cluster Engine ('1.2.7'): started and ready to provide service.
Jun 4 10:49:07 localhost corosync[23763]: [MAIN ] Successfully read main configuration file '/etc/corosync/corosync.conf'.

查看初始成员节点通知是否发出：
[root@node1 corosync]# grep -i totem /var/log/messages
Jun 4 10:42:30 localhost corosync[23702]: [TOTEM ] Initializing transport (UDP/IP).
Jun 4 10:42:30 localhost corosync[23702]: [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Jun 4 10:42:31 localhost corosync[23702]: [TOTEM ] The network interface [192.168.1.101] is now up.
Jun 4 10:42:31 localhost corosync[23702]: [TOTEM ] Process pause detected for 600 ms, flushing membership messages.
Jun 4 10:42:31 localhost corosync[23702]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
Jun 4 10:49:07 localhost corosync[23763]: [TOTEM ] Initializing transport (UDP/IP).
Jun 4 10:49:07 localhost corosync[23763]: [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Jun 4 10:49:08 localhost corosync[23763]: [TOTEM ] The network interface [192.168.1.101] is now up.
Jun 4 10:49:08 localhost corosync[23763]: [TOTEM ] A processor joined or left the membership and a new membership was formed.

检查过程中是否有错误产生：
[root@node1 corosync]# grep -i error: /var/log/messages |grep -v unpack_resources
没有提示是最好的结果；

检查pacemaker的启动状态：
[root@node1 corosync]# grep -i pcmk_startup /var/log/messages
Jun 4 10:42:31 localhost corosync[23702]: [pcmk ] info: pcmk_startup: CRM: Initialized
Jun 4 10:42:31 localhost corosync[23702]: [pcmk ] Logging: Initialized pcmk_startup
Jun 4 10:42:31 localhost corosync[23702]: [pcmk ] info: pcmk_startup: Maximum core file size is: 4294967295
Jun 4 10:42:31 localhost corosync[23702]: [pcmk ] info: pcmk_startup: Service: 9
Jun 4 10:42:31 localhost corosync[23702]: [pcmk ] info: pcmk_startup: Local hostname: node1.hanyu.com
Jun 4 10:49:08 localhost corosync[23763]: [pcmk ] info: pcmk_startup: CRM: Initialized
Jun 4 10:49:08 localhost corosync[23763]: [pcmk ] Logging: Initialized pcmk_startup
Jun 4 10:49:08 localhost corosync[23763]: [pcmk ] info: pcmk_startup: Maximum core file size is: 4294967295
Jun 4 10:49:08 localhost corosync[23763]: [pcmk ] info: pcmk_startup: Service: 9
Jun 4 10:49:08 localhost corosync[23763]: [pcmk ] info: pcmk_startup: Local hostname: node1.hanyu.com

node2：
[root@node2 corosync]# mkdir /var/lo
local/ lock/ log/
[root@node2 corosync]# mkdir /var/log/cluster
[root@node2 corosync]# corosync-keygen
Corosync Cluster Engine Authentication key generator.
Gathering 1024 bits for key from /dev/random.
Press keys on your keyboard to generate entropy.
Writing corosync key to /etc/corosync/authkey.

验证corosync引擎是否正常工作：
[root@node2 corosync]# service corosync start
Starting Corosync Cluster Engine (corosync): [ OK ]
[root@node2 corosync]# grep -i -e "corosync cluster engine" -e "configuration file" /var/log/messages
Jun 4 11:02:31 Eleven corosync[6203]: [MAIN ] Corosync Cluster Engine ('1.2.7'): started and ready to provide service.
Jun 4 11:02:31 Eleven corosync[6203]: [MAIN ] Successfully read main configuration file '/etc/corosync/corosync.conf'.

查看初始化成员节点通知是否发出：
[root@node2 corosync]# grep -i totem /var/log/messages
Jun 4 11:02:31 Eleven corosync[6203]: [TOTEM ] Initializing transport (UDP/IP).
Jun 4 11:02:31 Eleven corosync[6203]: [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Jun 4 11:02:31 Eleven corosync[6203]: [TOTEM ] The network interface [192.168.1.102] is now up.
Jun 4 11:02:34 Eleven corosync[6203]: [TOTEM ] A processor joined or left the membership and a new membership was formed.

检查过程中是有错误产生：
[root@node2 corosync]# grep -i error: /var/log/messages |grep -v unpack_resources

检查packmasker是否已经启动：
[root@node2 corosync]# grep -i pcmk_startup /var/log/messages
Jun 4 11:02:32 Eleven corosync[6203]: [pcmk ] info: pcmk_startup: CRM: Initialized
Jun 4 11:02:32 Eleven corosync[6203]: [pcmk ] Logging: Initialized pcmk_startup
Jun 4 11:02:33 Eleven corosync[6203]: [pcmk ] info: pcmk_startup: Maximum core file size is: 4294967295
Jun 4 11:02:33 Eleven corosync[6203]: [pcmk ] info: pcmk_startup: Service: 9
Jun 4 11:02:33 Eleven corosync[6203]: [pcmk ] info: pcmk_startup: Local hostname: node2.hanyu.com

(8)、
在节点node1
查看群集的状态：
[root@node1 corosync]# crm status
============
Last updated: Mon Jun 4 11:20:17 2012
Stack: openais
Current DC: node1.hanyu.com - partition with quorum
Version: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f
2 Nodes configured, 2 expected votes
0 Resources configured.
============

Online: [ node1.hanyu.com node2.hanyu.com ]

提供高可用性：

[root@node1 corosync]# crm configure show
node node1.hanyu.com
node node2.hanyu.com
property $id="cib-bootstrap-options" \
dc-version="1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
cluster-infrastructure="openais" \
expected-quorum-votes="2"

如何验证该文件的语法错误：
[root@node1 corosync]# crm_verify -L
crm_verify[23860]: 2012/06/04_11:25:46 ERROR: unpack_resources: Resource start-up disabled since no STONITH resources have been defined
crm_verify[23860]: 2012/06/04_11:25:46 ERROR: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option
crm_verify[23860]: 2012/06/04_11:25:46 ERROR: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity
Errors found during check: config not valid
-V may provide more details

禁用stonith：
[root@node1 corosync]# crm
crm(live)# configure
crm(live)configure# property stonith-enabled=false
crm(live)configure# commit
crm(live)configure#
检查一下状态：
crm(live)configure# show
node node1.hanyu.com
node node2.hanyu.com
property $id="cib-bootstrap-options" \
dc-version="1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
stonith-enabled="false"

再次检测：
[root@node1 corosync]# crm_verify –L
系统上有专门的stonith命令
【stonith -L 显示stonith所指示的类型
crm可以使用交互式模式
可以执行help
保存在cib里面，以xml的格式
集群的资源类型有4种
primitive 本地主资源 （只能运行在一个节点上）
group 把多个资源轨道一个组里面，便于管理
clone 需要在多个节点上同时启用的 （如ocfs2 ，stonith ，没有主次之分）
master 有主次之分，如drbd】

[root@node1 corosync]# crm
crm(live)# ra
crm(live)ra# classes
heartbeat
lsb
ocf / heartbeat linbit pacemaker
stonith
crm(live)ra# list lsb
NetworkManager acpid anacron
apmd atd auditd
autofs avahi-daemon avahi-dnsconfd
bluetooth capi conman
corosync cpuspeed crond
cups cups-config-daemon dc_client
dc_server dnsmasq drbd
dund firstboot functions
gpm haldaemon halt
heartbeat hidd hplip
httpd ip6tables ipmi
iptables irda irqbalance
isdn kdump killall
krb524 kudzu lm_sensors
logd lvm2-monitor mcstrans
mdmonitor mdmpd messagebus
microcode_ctl multipathd netconsole
netfs netplugd network
nfs nfslock nscd
ntpd openais openibd
pacemaker pand pcscd
portmap psacct rawdevices
rdisc readahead_early readahead_later
restorecond rhnsd rpcgssd
rpcidmapd rpcsvcgssd saslauthd
sendmail setroubleshoot single
smartd squid sshd
syslog tux vncserver
wdaemon winbind wpa_supplicant
xfs xinetd ypbind
yum-updatesd
crm(live)ra#
crm(live)ra#
crm(live)ra# list ocf heartbeat
AoEtarget AudibleAlarm CTDB
ClusterMon Delay Dummy
EvmsSCC Evmsd Filesystem
ICP IPaddr IPaddr2
IPsrcaddr IPv6addr LVM
LinuxSCSI MailTo ManageRAID
ManageVE Pure-FTPd Raid1
Route SAPDatabase SAPInstance
SendArp ServeRAID SphinxSearchDaemon
Squid Stateful SysInfo
VIPArip VirtualDomain WAS
WAS6 WinPopup Xen
Xinetd anything apache
conntrackd db2 drbd
eDir88 exportfs fio
iSCSILogicalUnit iSCSITarget ids
iscsi jboss ldirectord
mysql mysql-proxy nfsserver
nginx oracle oralsnr
pgsql pingd portblock
postfix proftpd rsyncd
scsi2reservation sfex syslog-ng
tomcat vmware

(9)、配置群集资源：

配置vip----资源ip
crm(live)configure# primitive webip ocf:heartbeat:IPaddr params ip=192.168.1.100
查看：
crm(live)configure# show
node node1.hanyu.com
node node2.hanyu.com
primitive webip ocf:heartbeat:IPaddr \
params ip="192.168.1.100"
property $id="cib-bootstrap-options" \
dc-version="1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
stonith-enabled="false"
提交
crm(live)configure# commit
[root@node1 ~]# crm
crm(live)# status
============
Last updated: Mon Jun 4 12:01:34 2012
Stack: openais
Current DC: node1.hanyu.com - partition with quorum
Version: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f
2 Nodes configured, 2 expected votes
1 Resources configured.
============

Online: [ node1.hanyu.com node2.hanyu.com ]

webip (ocf::heartbeat:IPaddr): Started node1.hanyu.com

在节点node1上查看实用ifconfig
[root@node1 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:E9:57:8C
inet addr:192.168.1.101 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fee9:578c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:159820 errors:0 dropped:0 overruns:0 frame:0
TX packets:61874 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:177495313 (169.2 MiB) TX bytes:7395778 (7.0 MiB)
Interrupt:67 Base address:0x2000

eth0:0 Link encap:Ethernet HWaddr 00:0C:29:E9:57:8C
inet addr:192.168.1.100 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:67 Base address:0x2000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:20040 errors:0 dropped:0 overruns:0 frame:0
TX packets:20040 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:14177363 (13.5 MiB) TX bytes:14177363 (13.5 MiB)

查看httpd的参数：
[root@node1 ~]# crm
crm(live)# ra
crm(live)ra# meta lsb:httpd
lsb:httpd

Apache is a World Wide Web server. It is used to serve \
HTML files and CGI.

Operations' defaults (advisory minimum):

start timeout=15
stop timeout=15
status timeout=15
restart timeout=15
force-reload timeout=15
monitor interval=15 timeout=15 start-delay=15

定义httpd的资源：
[root@node1 ~]# crm
crm(live)# configure
crm(live)configure# primitive webserver lsb:httpd
crm(live)configure# show
node node1.hanyu.com
node node2.hanyu.com
primitive webip ocf:heartbeat:IPaddr \
params ip="192.168.1.100"
primitive webserver lsb:httpd
property $id="cib-bootstrap-options" \
dc-version="1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
stonith-enabled="false"

查看状态：
crm(live)# status
============
Last updated: Mon Jun 4 12:18:23 2012
Stack: openais
Current DC: node1.hanyu.com - partition with quorum
Version: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f
2 Nodes configured, 2 expected votes
2 Resources configured.
============

Online: [ node1.hanyu.com node2.hanyu.com ]

webip (ocf::heartbeat:IPaddr): Started node1.hanyu.com
webserver (lsb:httpd): Started node2.hanyu.com

发现httpd已经启动，在node2节点上：

需要约束在同一节点上，定义成一个组
实用
[root@node1 ~]# crm
crm(live)# configure
crm(live)configure# help group

The `group` command creates a group of resources.

Usage:
...............
group <name> <rsc> [<rsc>...]
[meta attr_list]
[params attr_list]

attr_list :: [$id=<id>] <attr>=<val> [<attr>=<val>...] | $id-ref=<id>
...............
Example:
...............
group internal_www disk0 fs0 internal_ip apache \
meta target_role=stopped
...............

crm(live)configure#

crm(live)configure# group web webip webserver

crm(live)configure# show
node node1.hanyu.com
node node2.hanyu.com
primitive webip ocf:heartbeat:IPaddr \
params ip="192.168.1.100"
primitive webserver lsb:httpd
group web webip webserver
property $id="cib-bootstrap-options" \
dc-version="1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
stonith-enabled="false"

[root@node1 ~]# crm
crm(live)# status
============
Last updated: Mon Jun 4 12:32:03 2012
Stack: openais
Current DC: node1.hanyu.com - partition with quorum
Version: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f
2 Nodes configured, 2 expected votes
1 Resources configured.
============

Online: [ node1.hanyu.com node2.hanyu.com ]

Resource Group: web
webip (ocf::heartbeat:IPaddr): Started node1.hanyu.com
webserver (lsb:httpd): Started node1.hanyu.com

[root@node1 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:E9:57:8C
inet addr:192.168.1.101 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fee9:578c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:170656 errors:0 dropped:0 overruns:0 frame:0
TX packets:77145 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:178709996 (170.4 MiB) TX bytes:9320783 (8.8 MiB)
Interrupt:67 Base address:0x2000

eth0:0 Link encap:Ethernet HWaddr 00:0C:29:E9:57:8C
inet addr:192.168.1.100 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:67 Base address:0x2000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:20048 errors:0 dropped:0 overruns:0 frame:0
TX packets:20048 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:14177971 (13.5 MiB) TX bytes:14177971 (13.5 MiB)
使用pc机测试







[root@node2 ~]# cd /etc/corosync/

[root@node2 corosync]# crm status

============

Last updated: Mon Jun 4 12:40:01 2012
Stack: openais
Current DC: node2.hanyu.com - partition WITHOUT quorum
Version: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f
2 Nodes configured, 2 expected votes
1 Resources configured.
============

Online: [ node2.hanyu.com ]
OFFLINE: [ node1.hanyu.com ]

关闭quorum
[root@node1 corosync]# crm
crm(live)# configure
crm(live)configure# property no-quorum-policy=ignore

crm(live)configure# show
node node1.hanyu.com
node node2.hanyu.com
primitive webip ocf:heartbeat:IPaddr \
params ip="192.168.1.100"
primitive webserver lsb:httpd
group web webip webserver
property $id="cib-bootstrap-options" \
dc-version="1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
stonith-enabled="false" \
no-quorum-policy="ignore"
提交并退出；

[root@node2 corosync]# crm status
============
Last updated: Mon Jun 4 12:48:48 2012
Stack: openais
Current DC: node2.hanyu.com - partition WITHOUT quorum
Version: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f
2 Nodes configured, 2 expected votes
1 Resources configured.
============

Online: [ node2.hanyu.com ]
OFFLINE: [ node1.hanyu.com ]

Resource Group: web
webip (ocf::heartbeat:IPaddr): Started node2.hanyu.com
webserver (lsb:httpd): Started node2.hanyu.com







[root@node1 ~]# cd /etc/corosync/
[root@node1 corosync]# crm
crm crm_diff crm_master crm_node crm_resource crm_simulate crm_uuid crmadmin
crm_attribute crm_failcount crm_mon crm_report crm_shadow crm_standby crm_verify
corosync的常见指令
1，crm_attribute 修改集群的全局属性信息
比如前面的stonith和quorum
实际上是修改dc上的cib
2. crm_resource 修改资源
3. crm_node 管理节点
crm_node -e 查看节点的时代（配置文件修改过几次了）
[root@node2 corosync]# crm_node -q 显示当前节点的票数
1
4. cibadmin 集群配置的工具
-u, --upgrade Upgrade the configuration to the latest syntax
-Q, --query Query the contents of the CIB
-E, --erase Erase the contents of the whole CIB
-B, --bump Increase the CIB's epoch value by 1
如果某一个资源定义错了，就可以实用该工具进行删除
-D, --delete Delete the first object matching the supplied criteria, Eg.
也可以在crm的命令行下
crm(live)configure# delete
usage: delete <id> [<id>...]
也可以在该模式下执行edit
执行完毕后，commit 提交
5：集群的资源类型
crm(live)configure# help
- `primitive`本地主资源 （只能运行在一个节点上）
- `monitor`
- `group`
把多个资源轨道一个组里面，便于管理
- `clone` 需要在多个节点上同时启用的 （如ocfs2 ，stonith ，没有主次之分
- `ms`/`master` (master-slave) 有主次之分，如drbd
6：集群的资源代理ra可以用的类型
crm(live)configure ra# classes
heartbeat
lsb
ocf / heartbeat pacemaker
stonith
crm(live)configure ra# list heartbeat
AudibleAlarm Delay Filesystem ICP IPaddr IPaddr2 IPsrcaddr IPv6addr LVM
LinuxSCSI MailTo OCF Raid1 SendArp ServeRAID WAS WinPopup Xinetd
apache db2 hto-mapfuncs ids portblock
crm(live)configure ra# list lsb
NetworkManager acpid anacron apmd atd auditd
autofs avahi-daemon avahi-dnsconfd bluetooth capi conman
corosync cpuspeed crond cups cups-config-daemon dc_client
dc_server dnsmasq dund firstboot functions gpm
haldaemon halt heartbeat hidd hplip httpd
ip6tables ipmi iptables irda irqbalance isdn
kdump killall krb524 kudzu lm_sensors logd
lvm2-monitor mcstrans mdmonitor mdmpd messagebus microcode_ctl
multipathd mysqld netconsole netfs netplugd network
nfs nfslock nscd ntpd openais openibd
pacemaker pand pcscd portmap psacct rawdevices
rdisc readahead_early readahead_later restorecond rhnsd rpcgssd
rpcidmapd rpcsvcgssd saslauthd sendmail setroubleshoot single
smartd snmpd snmptrapd sshd syslog tgtd
vncserver vsftpd wdaemon winbind wpa_supplicant xfs
xinetd ypbind yum-updatesd
crm(live)configure ra# list ocf
AoEtarget AudibleAlarm CTDB ClusterMon Delay Dummy
EvmsSCC Evmsd Filesystem HealthCPU HealthSMART ICP
IPaddr IPaddr2 IPsrcaddr IPv6addr LVM LinuxSCSI
MailTo ManageRAID ManageVE Pure-FTPd Raid1 Route
SAPDatabase SAPInstance SendArp ServeRAID SphinxSearchDaemon Squid
Stateful SysInfo SystemHealth VIPArip VirtualDomain WAS
WAS6 WinPopup Xen Xinetd anything apache
conntrackd controld db2 drbd eDir88 exportfs
fio iSCSILogicalUnit iSCSITarget ids iscsi jboss
ldirectord mysql mysql-proxy nfsserver nginx o2cb
oracle oralsnr pgsql ping pingd portblock
postfix proftpd rsyncd scsi2reservation sfex syslog-ng
tomcat vmware

crm(live)configure ra# list stonith
apcmaster apcmastersnmp apcsmart baytech bladehpi
cyclades external/drac5 external/dracmc-telnet external/hmchttp external/ibmrsa
external/ibmrsa-telnet external/ipmi external/ippower9258 external/kdumpcheck external/rackpdu
external/riloe external/sbd external/vmware external/xen0 external/xen0-ha
fence_legacy ibmhmc ipmilan meatware nw_rpc100s
rcd_serial rps10 suicide wti_mpc

所需软件包下载地址：

本文出自 “小小屋，好好男人” 博客，转载请与作者联系！