Mysql cluster+heartbeat+ldirector+lvs 集群解决方案

数据库集群解决方案

原来考虑使用Mysql双主+LVS+keepalived+Ipvsadm,因为系统对数据一致性要求很高，双主复制存在此瓶颈，选择使用:Mysql CLuster +LVS+Heartbeat+Ldirector可很好的解决数据及时同步的问题并结合Heartbeat、Ldirector解决高可用、心跳检测、故障转移、负载均衡的需求

扩展:

Memcached:缓存服务，提升系统性能

Lucene:全文检索优化

扩展内容根据业务需求需要选择

数据库集群架构图：

略

Mysql cluster安装和配置略，各个服务器及节点启动方式如下：

10.0.0.21sql1、NDB1

1./var/lib/mysql/sbin/ndbd --initial

2../bin/mysqld_safe &

10.0.0.22sql2、NDB2

1./var/lib/mysql/sbin/ndbd --initial

2../bin/mysqld_safe &

10.0.0.24管理节点

./ndb_mgmd -f/var/lib/mysql-cluster/config.ini --initial

/var/lib/mysql/bin/ndb_mgm

ndb_mgmd> show

-------------------------------------------------------------

heartbeat安装与配置过程中出现的错误

1. ./configure --prefix=/usr/local/heatbeat

2.make 报错

（1）、找不到swing，解决方法：安装系统自带的swing RPM包

（2）、其他各种错误，去掉解压包里面Makefile文件中的-Werror参数

3.启动heatbeat时候报错

（1）Heartbeat not configured: /usr/local/heartbeat/etc/ha.d/ha.cf notfound.

修改/etc/rc.d/init.d/heartbeat 将上面错误中的路径改为/etc/ha.d/ha.cf

（2）/etc/init.d/heartbeat: line 48: /etc/ha.d/shellfuncs: No such fileor directory

缺少shellfuncs文件，安装cluster-glue RPM之后，安装后通过locate找到shellfuncs,把shellfuncs给cp到/etc/ha.d/

（3）cat: /usr/local/heartbeat/etc/ha.d/haresources: No such file ordirectory

locate找到该文件，cp到对应的路径即可

(4)[root@localhost dev]# service heartbeatstart

logd is already running

Starting High-Availability services:

Heartbeat failure [rc=139]. Failed.

错误很多，最终未完全处理，导致heartbeat无法开启，放弃此版本

换版本升级至最新的heartbeat 3.0.7

./ConfigureMe configure --prefix=XXXX报错

configure: error: in`/usr/local/heartbeat3.0.7':

configure: error: Core development headerswere not found

See `config.log' for more details.

./bootstrap exiting due to error (sorry!).

这需要中间层包Reusable-Cluster-Components-glue--glue-1.0.7.tar.bz2

#groupadd haclient

#useradd -g haclient hacluster

#wgethttp://hg.linux-ha.org/glue/archive/glue-1.0.7.tar.bz2

#tar jxvf glue-1.0.7.tar.bz2

#cdReusable-Cluster-Components-glue--glue-1.0.7/

#./autogen.sh

#./configure

报错:checking for special libxml2 includes... configure: error: libxml2config not found

缺少：libxml2-devel

安装rpm -ivh libxml2-devel-2.7.6-14.el6_5.2.x86_64.rpm

继续报错configure: error: BZ2 libraries not found

缺少bzip2

安装bzip2-devel-1.0.5-7.el6_0.x86_64.rpm

#make

#make install

./ConfigureMe configure --prefix=XXXX报错

configure: error: Core development headerswere not found

原因 缺少：resource-agents 在CSDN上下载

并安装

1354 tar -zxvf ClusterLabs-resource-agents-v3.9.2-0-ge261943.tar.gz

1355 ll

1356 cdClusterLabs-resource-agents-b735277/

1357 ll

1358 ./autogen.sh

1359 ./configure --prefix=/usr/local/heartbeat(安装在heartbeat同一个目录，特别重要否则还是会报上面的错误)

（启动heartbeat还是找不到相关文件：加参数重新configure

./configure --prefix=/usr/local/heartbeat--sysconfdir=/etc/ha.d libdir=/usr/local/heartbeat/lib64CFLAGS=-I/usr/local/heartbeat/includeLDFLAGS=L/usr/local/heartbeat/lib64LIBS='/lib64/libuuid.so.1'

启动heartbeat还是报错，cp Cluster-Resource-Agents-agents-1.0.3/heartbeat/shellfuncs/etc/ha.d

）

1360 make

1361 make install

make 错误1

gmake[4]: *** [ucast.lo] Error 1 解决方法:Heartbeat-3-0-7e3a82377fa8/lib/plugins/HBcomm/ucast.c

这个文件貌似有问题，删除468行int i = 1;应该就能make了。(这个真心坑爹)

gmake[1]: --xinclude: Command not found 缺少libxslt包：libxslt-1.1.26-2.el6_3.1.x86_64.rpm libxslt-devel-1.1.26-2.el6_3.1.x86_64.rpm

gmake[1]: Entering directory`/usr/local/heartbeat3.0.7/doc'

错误2/usr/bin/xsltproc \

--xinclude\
http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xslheartbeat.xml
error : connection refused

解决：因缺乏docbook-style-xsl-1.75.2-6.el6.noarch.rpm需安装及其依赖包

ERROR: Cannot open keyfile[/etc/ha.d/ha.d/authkeys

将/etc/ha.d/authkeys cp到/etc/ha.d/ha.d/即可

ERROR: Client child command[/usr/local/heartbeat/lib64/heartbeat/attrd] is not executable

做软链接 ln -s/usr/lib64/heartbeat/attrd /usr/local/heartbeat/lib64/heartbeat/attrd 其它类似

启动ldirector报错Can't locate LWP/UserAgent.pm in @IN

缺少perl-libwww-perl-5.833-2.el6.noarch.rpm

启动ldirector报错缺少依赖包：

处理：1951 rpm -ivhperl-devel-5.10.1-136.el6.x86_64.rpm --force --nodeps（强制覆盖安装）

1952 rpm -ivh perl-ExtUtils-CBuilder-0.27-136.el6.x86_64.rpm

1953 rpm -ivh perl-ExtUtils-MakeMaker-6.55-136.el6.x86_64.rpm

1954 ls*harness*

1955 ls*Harness*

1956 rpm -ivh perl-Test-Harness-3.17-136.el6.x86_64.rpm

1957 rpm -ivh perl-ExtUtils-MakeMaker-6.55-136.el6.x86_64.rpm

1958 history

启动继续报错Starting ldirectord... Can't locate URI.pm 安装perl-URI-1.40-2.el6.noarch.rpm

Starting ldirectord... Can't locateIO/Socket/INET6.pm安装perl-IO-Socket-INET6-2.56-4.el6.noarch.rpm

启动ldirector 报错找不到ldirector.cf 执行root@db1 ha.d]# cp /etc/ha.d/ldirectord.cf /usr/etc/ha.d/

查状态报错[root@db1 init.d]# ./ldirectord status

ldirectord stale pid

[root@db1 Packages]# ls *perl-DBI*

perl-DBI-1.609-4.el6.x86_64.rpm perl-DBIx-Simple-1.32-3.el6.noarch.rpm

[root@db1 Packages]# rpm -ivhperl-DBI-1.609-4.el6.x86_64.rpm

warning: perl-DBI-1.609-4.el6.x86_64.rpm:Header V3 RSA/SHA256 Signature, key ID c105b9de: NOKEY

Preparing... ########################################### [100%]

1:perl-DBI ###########################################[100%]

[root@db1 Packages]# ls *perl-DBD*

perl-DBD-MySQL-4.013-3.el6.x86_64.rpm perl-DBD-Pg-2.15.1-4.el6_3.x86_64.rpm perl-DBD-SQLite-1.27-3.el6.x86_64.rpm

[root@db1 Packages]# rpm -ivhperl-DBD-MySQL-4.013-3.el6.x86_64.rpm

warning:perl-DBD-MySQL-4.013-3.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key IDc105b9de: NOKEY

Preparing... ########################################### [100%]

1:perl-DBD-MySQL ########################################### [100%]

[root@db1 Packages]# cd -

/etc/rc.d/init.d

[root@db1 init.d]# ./ldirectord stop

Stopping ldirectord... success

[root@db1 init.d]# ./ldirectord start

Starting ldirectord... success

[root@db1 init.d]# ./ldirectord status

ldirectord for /usr/etc/ha.d/ldirectord.cfis running with pid: 25941 file /var/run/ldirectord.ldirectord.pid for/usr/etc/ha.d/ldirectord.cf

10.0.0.22 ldirector

启动报错[root@db2 init.d]# ./ldirectord start

Starting ldirectord... Can't locateMail/Send.pm in @INC

处理：rpm -ivh perl-MailTools-2.04-4.el6.noarch.rpm --force --nodeps （强制覆盖安装）

启动各种找不到配置文件ldirector.cf

处理方式：进入/etc/ha.d目录下用绝对路径去启动即可[root@db2 ha.d]# /etc/rc.d/init.d/ldirectord start

./configure --prefix=/usr/local/heartbeat--sysconfdir=/etc/ha.d libdir=/usr/local/heartbeat/lib64LIBS='/lib64/libuuid.so.1'

./configure --prefix=/usr/local/heartbeat--sysconfdir=/etc/ha.d libdir=/usr/local/heartbeat/lib64CFLAGS=-I/usr/local/heartbeat/include LDFLAGS=-L/usr/local/heartbeat/lib64LIBS='/lib64/libuuid.so.1'

./configure --prefix=/usr/local/heartbeat--sysconfdir=/etc/ha.d CFLAGS=-I/usr/local/heartbeat/includeLDFLAGS=-L/usr/local/heartbeat/lib64 LIBS='/lib64/libuuid.so.1'

sed -i's#/usr/lib/ocf#/usr/local/heartbeat/usr/lib/ocf#g' /etc/ha.d/shellfuncs

sed -i 's#/usr/lib/ocf#/usr/local/heartbeat/usr/lib/ocf#g'/etc/ha.d/resource.d/hto-mapfuncs

sed -i's#/usr/lib/ocf#/usr/local/heartbeat/usr/lib/ocf#g'/usr/local/heartbeat/usr/lib/ocf/lib/heartbeat/ocf-shellfuncs

10.0.0.22 heartbeat安装

安装

configure时候Reusable-Cluster-Components-glue--glue-1.0.7报错configure:error: You need glib2-devel installed in order to build cluster-glue

缺少glib2-devel

make 时候 Entering directory`/usr/local/Reusable-Cluster-Components-glue--glue-1.0.7/libltdl'

最后

在节点1上[root@db1 sbin]# mysql -uldirector -p -h 10.0.0.25

建库建表create table MyClass(id int(4));

mysql> show databases;

+--------------------+

| Database |

+--------------------+

| information_schema |

| ldirectordb |

| test |

+--------------------+

3 rows in set (0.02 sec)

mysql> show tables;

ERROR 1046 (3D000): No database selected

mysql> use ldirectordb

Database changed

mysql> show tables;

+-----------------------+

| Tables_in_ldirectordb |

+-----------------------+

| connectioncheck |

| myclass |

+-----------------------+

2 rows in set (0.05 sec)

再到节点2上看发现表数据同步过来了[root@db2 ha.d]# mysql -u ldirector -p -h 10.0.0.25

mysql> show databases;

+--------------------+

| Database |

+--------------------+

| information_schema |

| ldirectordb |

| test |

+--------------------+

3 rows in set (0.00 sec)

mysql> use ldirectordb

Database changed

mysql> show tables;

+-----------------------+

| Tables_in_ldirectordb |

+-----------------------+

| connectioncheck |

| myclass |

+-----------------------+

2 rows in set (0.03 sec)

最后 Web连接虚拟ip 10.0.0.25:3306

关闭其中一个节点服务器，发现数据库连接还可以存活