MongoDB高可用架构搭建及应用

最近公司在使用mongodb，现在先把mongodb的高可用集群贴出来，共大家分享，今后一起讨论.

特点：非关系型的、分布式的、开源的、水平可扩展的。

MongoDB高可用架构的搭建

分片存储服务器规划

每个分片3服务器，前期采用三台，日后服务器的增加考虑灾备，服务增加的基数最少为两台。

类型 服务器 用途 系统 说明

存储/数据

Server1 Shard1/Shard2/Shard3 Linux 64位 Shard1:10001;Shard2:10002,Shard3:10003;

Server2 Shard1/Shard2/Shard3 Linux 64位 Shard1:10001;Shard2:10002,Shard3:10003;

Server3 Shard1/Shard2/Shard3 Linux 64位 Shard1:10001;Shard2:10002,Shard3:10003;

配置

Server1 Config1 Linux 64位 Config1:20000;

Server2 Config2 Linux 64位 Config2:20000;

Server3 Config3 Linux 64位 Config3:20000;

路由

Server1 Mongos1 Linux 64位 Mongos:30000

Server2 Mongos2 Linux 64位 Mongos:30000

Server3 Mongos3 Linux 64位 Mongos:30000

准备

创建配置、日志、分片、key文件存储目录及验证文件

[root@Mongo-server-B /]# mkdir /data/config/ -p

[root@Mongo-server-B /]# mkdir /data/log/ -p

[root@Mongo-server-B /]# mkdir /data/shard1/ -p

[root@Mongo-server-B /]# mkdir /data/shard2/ -p

[root@Mongo-server-B /]# mkdir /data/shard3/ -p

[root@Mongo-server-B /]# mkdir /data/key/ -p

创建验证与无验证目录

[root@Mongo-server-B /]# mkdir /Apps/mongo/bin/nosecurity/ -p

[root@Mongo-server-B /]# mkdir /Apps/mongo/bin/security/ -p

创建配置文件

1，创建验证文件security于/data /key/目录，关赋予可读权限，命令如下：

[root@Mongo-server-B /]# cd /data/key/

[root@Mongo-server-B key]# echo 'pomohoshard1key'> security

[root@Mongo-server-B key]# chmod 600 security

2，创建shard1.conf、shard2.conf、shard3.conf、configsvr.conf、mongos.conf于/Apps/mongo/bin/nosecurity目录与/Apps/mongo/bin/security目录，内容分别如下：

shard1.conf

dbpath = /data/shard1

shardsvr = true

replSet = shard1

bind_ip = 192.168.2.88,localhost

port = 10001

oplogSize = 100

logpath = /data/log/shard1.log

logappend = true

profile = 1

slowms = 5

rest = true

fork = true

keyFile = /data/key/security #nosecurity目录将该行删除

shard2.conf

dbpath = /data/shard2

shardsvr = true

replSet = shard2

bind_ip = 192.168.2.88,localhost

port = 10002

oplogSize = 100

logpath = /data/log/shard2.log

logappend = true

profile = 1

slowms = 5

rest = true

fork = true

keyFile = /data/key/security #nosecurity目录将该行删除

shard3.conf

dbpath = /data/shard3

shardsvr = true

replSet = shard3

bind_ip = 192.168.2.88,localhost

port = 10003

oplogSize = 100

logpath = /data/log/shard3.log

logappend = true

profile = 1

slowms = 5

rest = true

fork = true

keyFile = /data/key/security #nosecurity目录将该行删除

configsvr.conf

dbpath = /data/config

configsvr = true

port = 20000

logpath = /data/log/config.log

logappend = true

fork = true

keyFile = /data/key/security #nosecurity目录将该行删除

mongos.conf

configdb = 192.168.2.88:20000, 192.168.2.89:20000, 192.168.2.90:20000

port = 30000

chunkSize = 5

logpath = /data/log/mongos.log

logappend = true

fork = true

keyFile = /data/key/security #nosecurity目录将该行删除

分片配置

说明：分片要在无验证环境中配置，否则会出现无权限等异常。采用以下命令启动Server1\Server2\Server3上的shard1\shard2\shard3：

[root@Mongo-server-A bin]# cd /Apps/mongo/bin/

[root@Mongo-server-A bin]# ./mongod -f ./nosecurity/shard1.conf

[root@Mongo-server-A bin]# ./mongod -f ./nosecurity/shard2.conf

[root@Mongo-server-A bin]# ./mongod -f ./nosecurity/shard3.conf

以下命令查看是否正常启动：

[root@Mongo-server-A bin]# netstat –lnpt

启动后连接到shard1\shard2\shard3分别进行配置，以下是具体配置过程：

[root@Mongo-server-A bin]# ./mongo 192.168.2.88:10001

>config = {_id: "shard1", members: [

{_id: 0, host: "192.168.2.88:10001"},

{_id: 1, host: "192.168.2.89:10001"},

{_id: 2, host: "192.168.2.90:10001"}]

}

> rs.initiate(config)

>exit

[root@Mongo-server-A bin]# ./mongo 192.168.2.88:10002

>config = {_id: "shard2", members: [

{_id: 0, host: "192.168.2.88:10002"},

{_id: 1, host: "192.168.2.89:10002"},

{_id: 2, host: "192.168.2.90:10002"}]

}

> rs.initiate(config)

>exit

[root@Mongo-server-A bin]# ./mongo 192.168.2.88:10003

>config = {_id: "shard3", members: [

{_id: 0, host: "192.168.2.88:10003"},

{_id: 1, host: "192.168.2.89:10003"},

{_id: 2, host: "192.168.2.90:10003"}]

}

> rs.initiate(config)

至此，已完成分片配置

路由设置

路由是能过config来连接分片服务器，在启动路由进程时，先启动配置进程，路由配置过程如下：

[root@Mongo-server-A bin]# ./mongod -f ./nosecurity/configsvr.conf

[root@Mongo-server-A bin]# ./mongos -f ./nosecurity/mongos.conf

启动后，连接路由进行分片添加，只需配置一台路由。注：分片操作需在admin库下进行，另外必需在无验证要求下进行，即采用前面创建于nosecurity文件夹下的配置。

[root@Mongo-server-A bin]# ./mongo 192.168.2.88:30000

mongos> use admin

mongos> db.runCommand( {addshard:"shard1/192.168.2.88:10001,192.168.2.89:10001,192.168.2.90:10001", name:"shard1", maxsize:20480} )

mongos> db.runCommand( {addshard:"shard2/192.168.2.88:10002,192.168.2.89:10002,192.168.2.90:10002", name:"shard2", maxsize:20480} )

mongos> db.runCommand( {addshard:"shard3/192.168.2.88:10003,192.168.2.89:10003,192.168.2.90:10003", name:"shard3", maxsize:20480} )

命令检查分片添加情况，如出现以下结果则表示配置成功：

mongos> db.runCommand( { listshards : 1 } )

{

"shards" : [

{

"_id" : "shard1",

"host" : "shard1/192.168.2.88:10001,192.168.2.89:10001,192.168.2.90:10001"

},

{

"_id" : "shard2",

"host" : "shard2/192.168.2.88:10002,192.168.2.89:10002,192.168.2.90:10002"

},

{

"_id" : "shard3",

"host" : "shard3/192.168.2.88:10003,192.168.2.89:10003,192.168.2.90:10003"

}

],

"ok" : 1

}

权限控制

MongoDB默认为验证模式。如需对数据库进行权限控制，需先采用无验证模式登录，进入admin库创建管理员用户后，再采用验证模式登录。通过前面创建的管理员帐号进行数据库与用户的创建。MongoDB集群的权限与单台的权限控制的不同之处在于，单台是通过-auth属性，集群是通过keyFile来进行服务器间的验证。以下介绍配置全过程。

前面的所有步骤，都是在nosecurity模式下进行。如果没有采用非验证模式的需要将所有进程（分片、配置、mongos）停止，将切换到无验证模式。

步骤一：先进行登录，并切换进admin库创建管理员帐号

[root@Mongo-server-A bin]# ./mongo 192.168.2.88:30000

mongos> use admin

mongos> db.addUser('admin','123456')

{

"singleShard" : "192.168.2.88:20000,192.168.2.89:20000,192.168.2.90:20000",

"n" : 0,

"connectionId" : 211,

"err" : null,

"ok" : 1

}

{

"_id" : ObjectId("4f6c78ddad912a3ac6833ece"),

"user" : "admin",

"readOnly" : false,

"pwd" : "95ec4261124ba5951720b199908d892b"

}

验证用户名与密码

mongos> db.auth('admin','123456')

1

mongos>exit

步骤二：退出后，将Server1\Server2\Server3服务器上MongoDB的所有进程（分片、配置、mongos）停止，将切换到验证模式。具体命令如下：

[root@Mongo-server-A bin]# killall mongod mongos

[root@Mongo-server-A bin]# netstat -lnpt

[root@Mongo-server-A bin]# ./mongod -f ./security/shard1.conf

[root@Mongo-server-A bin]# ./mongod -f ./security/shard2.conf

[root@Mongo-server-A bin]# ./mongod -f ./security/shard3.conf

[root@Mongo-server-A bin]# netstat –lnpt

[root@Mongo-server-A bin]# ./mongod -f ./security/configsvr.conf

[root@Mongo-server-A bin]# ./mongos -f ./security/mongos.conf

启动后，如对库进行查看，则会报以下异常：

[root@Mongo-server-A bin]# ./mongo 192.168.2.90:30000/admin

MongoDB shell version: 2.0.3

connecting to: 192.168.2.90:30000/admin

> show dbs

Fri Mar 23 22:28:28 uncaught exception: listDatabases failed:{ "ok" : 0, "errmsg" : "unauthorized" }

以下是正常登录后显示的信息：

[root@Mongo-server-A bin]# ./mongo 192.168.2.90:30000/admin

MongoDB shell version: 2.0.3

connecting to: 192.168.2.90:30000/admin

> db.auth('admin','123456')

1

mongos>

步骤三：以下是数据库及数据库用户创建的过程：

mongos> use hello

switched to db hello

mongos> db.addUser('sa','sa')

{

"singleShard" : "shard2/192.168.2.88:10002,192.168.2.89:10002,192.168.2.90:10002",

"n" : 0,

"lastOp" : NumberLong("5723101431532093441"),

"connectionId" : 38,

"err" : null,

"ok" : 1

}

{

"user" : "sa",

"readOnly" : false,

"pwd" : "75692b1d11c072c6c79332e248c4f699",

"_id" : ObjectId("4f6c8a6e9f67b049a20a00de")

}

mongos> exit

bye

[root@Mongo-server-A bin]# ./mongo 192.168.2.90:30000/hello -u sa -p

MongoDB shell version: 2.0.3

Enter password:

connecting to: 192.168.2.90:30000/hello

> show collections

system.indexes

system.users

> db.system.users.find()

{ "_id" : ObjectId("4f6c8a6e9f67b049a20a00de"), "user" : "sa", "readOnly" : false, "pwd" : "75692b1d11c072c6c79332e248c4f699" }

创建数据库

> use pomoho

> db.addUser("pomoho", "******")

> db.auth("pomoho","******")

> use admin

> db.runCommand( { enablesharding : "pomoho"} )

> db.runCommand( { shardcollection : "pomoho.tablename", key : {primarykey : 1} } )

> use pomoho

> db.stats()

#设置从库可查询

db.getMongo().setSlaveOk()

rs.setSlaveOk()

show collections

db.c1.insert({age:30})

db.c1.find()

分片设置：db.runCommand( { addshard : "shard1", maxSize:500000} );
本文出自 “未来” 博客，请务必保留此出处http://jiangjm.blog.51cto.com/3101948/1088875