【原】ubuntu下Mysql的HA（corosync+pacemaker+drbd）

一、前提准备：

1、OS：ubuntu 12.04

2、cat /etc/hosts：

127.0.0.1 localhost
192.168.153.154 ha1
192.168.153.155 ha2

3、硬盘：/dev/sdb1

4、vip：192.168.153.200

二、corosync和pacemaker的安装配置：

1、[ha1+ha2]：

apt-get install -y pacemaker corosync

2、[ha1]：

1 corosync-keygen（时间较长）
2 scp /etc/corosync/authkey root@ha2:/etc/corosync/

3、[ha1+ha2]：取消corosync和pacemaker的开机启动：

apt-get install -y sysv-rc-conf
sysv-rc-conf（使用空格把drbd,pacemaker,corosync有X的地方取消掉，q退出）
sed -i 's/127.0.0.1/192.168.153.0/g' /etc/corosync/corosync.conf
sed -i 's/ver:       0/ver:       1/g' /etc/corosync/corosync.conf
mkdir /var/log/cluster
update-rc.d pacemaker start 50 1 2 3 4 5 . stop 01 0 6 .
sed -i 's/no/yes/g' /etc/default/corosync
service corosync start
corosync-cfgtool -s
corosync-quorumtool -l
service pacemaker start
验证:crm status

root@ha1:~# crm status
============
Last updated: Tue Feb 25 22:18:44 2014
Last change: Tue Feb 25 22:17:49 2014 via crmd on ha1
Stack: openais
Current DC: ha1 - partition with quorum
Version: 1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c
2 Nodes configured, 2 expected votes
0 Resources configured.
============

Online: [ ha1 ha2 ]

三、drbd的安装配置：

1、[ha1+ha2]：drbd的安装：

tar -zxvf drbd8_8.4.4.orig.tar.gz（由于drbd8.4.2有bug，所以这里选择手动安装8.4.4版本）
apt-get install -y debhelper flex xsltproc
cd drbd8_8.4.4
./configure --localstatedir=/var --sysconfdir=/etc --with-km
make && make install

2、[ha1+ha2]：创建磁盘：

fdisk /dev/sdb
n
p
1
2048
...
w

3、[ha1+ha2]：创建drbd的mysql资源：

cat /etc/drbd.d/mysql.res
resource mysql {
device /dev/drbd0;
disk /dev/sdb1;
meta-disk internal;
on ha1 {
address 192.168.153.154:7788;
}
on ha2 {
address 192.168.153.155:7788;
}
syncer {
rate 10M;
}
}
echo -e 'drbd # -> added' >> /etc/modules
drbdadm dump mysql
drbdadm create-md mysql
drbdadm up mysql
验证：drbd-overview
0:mysql/0  Connected Secondary/Secondary Inconsistent/Inconsistent C r-----

4、[ha1]：

drbdadm -- --overwrite-data-of-peer primary mysql
cat /proc/drbd：
root@ha1:~# cat /proc/drbd
version: 8.4.4 (api:1/proto:86-101)
GIT-hash: 599f286440bd633d15d5ff985204aff4bccffadd build by root@ha1, 2014-02-25 22:22:00
0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
ns:108044 nr:0 dw:0 dr:108708 al:0 bm:6 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:20861776
[>....................] sync'ed:  0.6% (20372/20476)M
finish: 0:16:02 speed: 21,608 (21,608) K/sec
直到同步100%的时候才成功。结果是：
root@ha1:~# cat /proc/drbd
version: 8.4.4 (api:1/proto:86-101)
GIT-hash: 599f286440bd633d15d5ff985204aff4bccffadd build by root@ha1, 2014-02-25 22:22:00
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:20969820 nr:0 dw:0 dr:20970484 al:0 bm:1280 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
service drbd restart
mkfs.ext4 /dev/drbd0
mkdir /mydata
mount /dev/drbd0 /mydata
mkdir -p /mydata/testfile
umount /dev/drbd0
drbdadm secondary mysql

5、[ha2]：

service drbd restart
drbdadm primary mysql
mkdir /mydata
mount /dev/drbd0 /mydata
ll /mydata/ 如果可以看到testfile说明成功。
umount /dev/drbd0
drbdadm secondary mysql（如果中间出现错误，可以把ha1和ha2的drbd重启即可。）

6、[ha1]:

drbdadm primary mysql

7、[ha1+ha2]：mysql的安装配置

apt-get install -y mysql-server
sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf
service mysql restart
mysql -uroot -p
grant all privileges on *.* to root@"%" identified by "password" with grant option;
FLUSH PRIVILEGES;
delete from mysql.user where user="";
quit;
service mysql stop

8、[ha1]:

drbdadm primary mysql
drbd-overview
mount /dev/drbd0 /mydata
cp -R -p /var/lib/mysql /mydata/

9、[ha1+ha2]:

sed -i 's/2345/!0123456/g' /etc/init/mysql.conf
sed -i -e '/datadir/cdatadir         = \/mydata\/mysql'  /etc/mysql/my.cnf
sed -i '/\/var\/lib\/mysql\/ /c\/mydata\/mysql\/ r,' /etc/apparmor.d/usr.sbin.mysqld
sed -i '/\/var\/lib\/mysql\/\*\* rwk/c\/mydata\/mysql\/** rwk,' /etc/apparmor.d/usr.sbin.mysqld

10、[ha1]:

service mysql start
mysql -uroot -p
CREATE DATABASE test1;
quit
service mysql stop
umount /dev/drbd0
drbdadm secondary mysql

11、[ha2]:

drbdadm primary mysql
mount /dev/drbd0 /mydata/
service mysql start
查看是否有数据库test1
service mysql stop
umount /dev/drbd0
drbdadm secondary mysql

12、[ha1]:

drbdadm primary mysql

13、[ha1+ha2]:

service drbd stop

14、[ha1]:

crm configure property stonith-enabled=false
crm configure property no-quorum-policy=ignore

crm configure
primitive mysqldrbd ocf:heartbeat:drbd params drbd_resource=mysql op start timeout=240 op stop timeout=100 op monitor role=Master interval=20 timeout=30 op monitor role=Slave interval=30 timeout=30
ms ms_mysqldrbd mysqldrbd meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true
commit
exit
验证:crm status
root@ha1:~# crm status
============
Last updated: Tue Feb 25 23:06:09 2014
Last change: Tue Feb 25 23:06:05 2014 via cibadmin on ha1
Stack: openais
Current DC: ha1 - partition with quorum
Version: 1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c
2 Nodes configured, 2 expected votes
2 Resources configured.
============

Online: [ ha1 ha2 ]

Master/Slave Set: ms_mysqldrbd [mysqldrbd]
Masters: [ ha1 ]
Slaves: [ ha2 ]

crm configure
primitive mystore ocf:heartbeat:Filesystem params device=/dev/drbd0 directory=/mydata fstype=ext4 op start timeout=60 op stop timeout=60
colocation mystore_with_ms_mysqldrbd inf: mystore ms_mysqldrbd:Master
order mystore_after_ms_mysqldrbd mandatory: ms_mysqldrbd:promote mystore:start
commit
exit
验证:crm status,mount
Master/Slave Set: ms_mysqldrbd [mysqldrbd]
Masters: [ ha1 ]
Slaves: [ ha2 ]
mystore (ocf::heartbeat:Filesystem): Started ha1

crm configure
primitive mysql lsb:mysql
colocation mysql_with_mystore inf: mysql mystore
order mysql_after_mystore mandatory: mystore mysql
commit
exit
验证:crm status
Master/Slave Set: ms_mysqldrbd [mysqldrbd]
Masters: [ ha1 ]
Slaves: [ ha2 ]
mystore (ocf::heartbeat:Filesystem): Started ha1
mysql (lsb:mysql): Started ha1

crm configure
primitive vip ocf:heartbeat:IPaddr params ip=192.168.153.200 nic=eth0 cidr_netmask=255.255.255.0
colocation vip_with_ms_mysqldrbd inf: ms_mysqldrbd:Master vip
commit
exit
验证crm status
Master/Slave Set: ms_mysqldrbd [mysqldrbd]
Masters: [ ha1 ]
Slaves: [ ha2 ]
mystore (ocf::heartbeat:Filesystem): Started ha1
mysql (lsb:mysql): Started ha1
vip (ocf::heartbeat:IPaddr): Started ha1

四、验证：

1、客户端mysql验证：

mysql -uroot -p -h 192.168.153.200
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
| test1              |
+--------------------+

2、模拟故障：

root@ha1:~# crm
crm(live)# node
crm(live)node# standby
crm(live)node# show
ha1: normal
standby: on
ha2: normal
crm(live)node# exit
bye
此时看一下状态：
root@ha1:~# crm status
Node ha1: standby
Online: [ ha2 ]

Master/Slave Set: ms_mysqldrbd [mysqldrbd]
Masters: [ ha2 ]
Stopped: [ mysqldrbd:0 ]
mystore (ocf::heartbeat:Filesystem): Started ha2
mysql (lsb:mysql): Started ha2
vip (ocf::heartbeat:IPaddr): Started ha2

此时，mysql已经运行于ha2上了。