KVM虚拟化开源高可用方案(三)glusterfs 推荐

说明：
设计目的：
1. 集群设计虚拟机容量70-100台,占用1个机柜，全部由1U服务器组成，其中存储服务器6台，300G*8，节点服务器10台；
2. 虚拟机可以在KVM集群宿主机之间迁移；
glusterfs集群架构：
1. 存储服务器和节点服务器组成，存储服务器通过哈希算法，可以弹性增加或者减少，并实现冗余；
2. 存储服务器每台机器至少需要4块网卡，如果机器只有板载的2块网卡，需要在加1块双口网卡，做4块网卡的绑定，这样可以提高网络带宽；
3. KVM集群每台宿主机作为glusterfs客户端，挂载glusterfs集群的文件系统，将虚拟机放置在上面；
glusterfs集群搭建：
存储节点搭建
1 安装相关组件
yum -y install wget fuse fuse-libs
2 下载安装测试
yum -y install wget fuse fuse-libs
wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-debuginfo-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-devel-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-fuse-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-geo-replication-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-rdma-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-server-3.3.0-1.el6.x86_64.rpm yum install glusterfs-* -y
/usr/sbin/glusterfs -V
3 启动glusterd服务
默认安装完成自动启动服务，安装完成后是一个标准的redhat服务，可以启动，停止，重启
/etc/init.d/glusterd start
/etc/init.d/glusterd stop
/etc/init.d/glusterd restart
chkconfig glusterd on
4 添加服务器到存储池
在第一台服务器上执行探测操作
gluster peer probe server1
gluster peer probe server2
gluster peer probe server3
gluster peer probe server4
gluster peer probe server5
校验集群状态
[root@hp246 ~]# gluster peer status
Number of Peers: 4
Hostname: server1
Uuid: 59cd74a9-a555-4560-b98e-a7eaf2058926
State: Peer in Cluster (Connected)
Hostname: server2
Uuid: 278d94f8-cf55-42cc-a4ad-9f84295c140b
State: Peer in Cluster (Connected)
Hostname: server3
Uuid: 7fd840a2-53f5-4540-b455-3e5e7eded813
State: Peer in Cluster (Connected)
Hostname: server4
Uuid: 4bfd8649-7f74-4a9f-9f04-4267cc80a1c3
State: Peer in Cluster (Connected)
…
如果需要移出集群 执行如下命令
gluster peer detach server
5 创建集群卷
1)创建一个分布卷（只是卷连接起来，跨区卷）
gluster volume create test-volume server1:/exp1 server2:/exp2 server3:/exp3 server4:/
exp4
2)创建一个复制卷(类似raid1)
gluster volume create test-volume replica 2 transport tcp server1:/exp1 server2:/exp2
3)创建一个条带卷(类似raid0)
gluster volume create test-volume stripe 2 transport tcp server1:/exp1 server2:/exp2
4）创建一个分布条带卷(类似raid0+0)
gluster volume create test-volume stripe 4 transport tcp server1:/exp1 server2:/exp2
server3:/exp3 server4:/exp4 server5:/exp5 server6:/exp6 server7:/exp7 server8:/exp8
5）创建一个复制条带卷(类似raid1然后跨区)
gluster volume create test-volume replica 2 transport tcp server1:/exp1 server2:/exp2
server3:/exp3 server4:/exp4
6)创建一个分部复制条带卷(类似raid10)
gluster volume create test-volume stripe 2 replica 2 transport tcp server1:/exp1
server2:/exp2 server3:/exp3 server4:/exp4 server5:/exp5 server6:/exp6 server7:/exp7
server8:/exp8
7)创建条带复制卷(类似raid01)
gluster volume create test-volume stripe 2 replica 2 transport tcp server1:/exp1
server2:/exp2 server3:/exp3 server4:/exp4
查看卷信息
gluster volume info
启动卷
gluster volume start test-volume
6 客户端挂载
modprobe fuse
Verify that the FUSE module is loaded:
# dmesg | grep -i fuse
fuse init (API version 7.13)
yum -y install wget fuse fuse-libs
wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-debuginfo-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-devel-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-fuse-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-geo-replication-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-rdma-3.3.0-1.el6.x86_64.rpm wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-server-3.3.0-1.el6.x86_64.rpm yum install glusterfs-* -y
挂载卷
mount -t glusterfs server1:/test-volume /mnt/glusterfs
mount -t glusterfs hp246:/test-volume /gfs
自动挂载
vim fstab
server1:/test-volume /mnt/glusterfs glusterfs defaults,_netdev 0 0
创建虚拟机：略

欢迎加我微信（ID：xiaolikvm）一起交流。欢迎加入云技术社区KVM/QEMU技术交流QQ，请加QQ:502207183，注明姓名，城市，及要加入KVM群。