您的位置:首页 > 数据库 > Mongodb

MongoDB副本集的搭建

2018-02-02 17:50 357 查看
原文地址:http://www.cnblogs.com/ivictor/p/6795418.html

副本集是mongodb提供的一种高可用解决方案。相对于原来的主从复制,副本集能自动感知primary节点的下线,并提升其中一个Secondary作为Primary。

整个过程对业务透明,同时也大大降低了运维的成本。

架构图如下:



 

MongoDB副本集的角色

1. Primary

   默认情况下,读写都是在Primary上操作的。

2. Secondary

   通过oplog来重放Primary上的所有操作,拥有Primary节点数据的完整拷贝。

   默认情况下,不可写,也不可读。

   根据不同的需求,Secondary又可配置为如下形式:

   1> Priority 0 Replica Set Members

        优先级为0的节点,优先级为0的成员永远不会被选举为primary。

        在mongoDB副本集中,允许给不同的节点设置不同的优先级。

        优先级的取值范围为0-1000,可设置为浮点数,默认为1。

        拥有最高优先级的成员会优先选举为primary。

        譬如,在副本集中添加了一个优先级为2的成员node3:27020,而其它成员的优先级为1,只要node3:27020拥有最新的数据,那么当前的primary就会自动降

        级,node3:27020将会被选举为新的primary节点,但如果node3:27020中的数据不够新,则当前primary节点保持不变,直到node3:27020的数据更新到最新。

    2> Hidden Replica Set Members-隐藏节点

         隐藏节点的优先级同样为0,同时对客户端不可见

         使用rs.status()和rs.config()可以看到隐藏节点,但是对于db.isMaster()不可见。客户端连接到副本集时,会调用db.isMaster()命令来查看可用成员信息。

         所以,隐藏节点不会受到客户端的读请求。

         隐藏节点常用于执行特定的任务,譬如报表,备份。

    3> Delayed Replica Set Members-延迟节点

         延迟节点会比primary节点延迟指定的时间(通过slaveDelay参数来指定)

         延迟节点必须是隐藏节点。

3. Arbiter

   仲裁节点,只是用来投票,且投票的权重只能为1,不复制数据,也不能提升为primary。

   仲裁节点常用于节点数量是偶数的副本集中。

   建议:通常将Arbiter部署在业务服务器上,切忌将其部署在Primary节点或Secondary节点服务器上。

 

注:一个副本集最多有50个成员节点,7个投票节点。

 

MongoDB副本集的搭建

创建数据目录

# mkdir -p /data/27017

# mkdir -p /data/27018

# mkdir -p /data/27019

为了便于查看运行过程中的日志信息,为每个实例创建单独的日志文件

# mkdir -p /var/log/mongodb/

 

启动mongod实例

# mongod --replSet myapp --dbpath /data/27017 --port 27017 --logpath /var/log/mongodb/27017.log --fork

# mongod --replSet myapp --dbpath /data/27018 --port 27018 --logpath /var/log/mongodb/27018.log --fork

# mongod --replSet myapp --dbpath /data/27019 --port 27019 --logpath /var/log/mongodb/27019.log --fork

以27017端口实例为例,其日志输出信息如下:



2017-05-02T14:05:22.745+0800 I CONTROL  [initandlisten] MongoDB starting : pid=2739 port=27017 dbpath=/data/27017 64-bit host=node3
2017-05-02T14:05:22.745+0800 I CONTROL  [initandlisten] db version v3.4.2
2017-05-02T14:05:22.745+0800 I CONTROL  [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b
2017-05-02T14:05:22.745+0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2017-05-02T14:05:22.745+0800 I CONTROL  [initandlisten] allocator: tcmalloc
2017-05-02T14:05:22.745+0800 I CONTROL  [initandlisten] modules: none
2017-05-02T14:05:22.745+0800 I CONTROL  [initandlisten] build environment:
2017-05-02T14:05:22.745+0800 I CONTROL  [initandlisten]     distmod: rhel62
2017-05-02T14:05:22.745+0800 I CONTROL  [initandlisten]     distarch: x86_64
2017-05-02T14:05:22.745+0800 I CONTROL  [initandlisten]     target_arch: x86_64
2017-05-02T14:05:22.745+0800 I CONTROL  [initandlisten] options: { net: { port: 27017 }, processManagement: { fork: true }, replication: { replSet: "myapp" }, storage: { dbPath: "/data/27017" }, systemLog: { destination: "file", path: "/var/log/mongodb/27017.log" } }
2017-05-02T14:05:22.768+0800 I -        [initandlisten] 2017-05-02T14:05:22.768+0800 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine2017-05-02T14:05:22.768+0800 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem 2017-05-02T14:05:22.769+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=256M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),2017-05-02T14:05:24.450+0800 I CONTROL  [initandlisten]
2017-05-02T14:05:24.482+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-05-02T14:05:24.482+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-05-02T14:05:24.482+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.2017-05-02T14:05:24.482+0800 I CONTROL  [initandlisten]
2017-05-02T14:05:24.516+0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/data/27017/diagnostic.data'2017-05-02T14:05:24.517+0800 I REPL     [initandlisten] Did not find local voted for document at startup.
2017-05-02T14:05:24.517+0800 I REPL     [initandlisten] Did not find local replica set configuration document at startup;  NoMatchingDocument: Did not find replica set configuration document in local.system.replset
2017-05-02T14:05:24.519+0800 I NETWORK  [thread1] waiting for connections on port 27017


 

通过mongo连接副本集任一成员,在这里,连接27017端口实例

# mongo

初始化副本集

> rs.initiate()
{
"info2" : "no configuration specified. Using a default configuration for the set",
"me" : "node3:27017",
"ok" : 1
}


可通过rs.conf()查看当前副本集的配置信息,

myapp:PRIMARY> rs.conf()
{
"_id" : "myapp",
"version" : 1,
"protocolVersion" : NumberLong(1),
"members" : [
{
"_id" : 0,
"host" : "node3:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {

},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : 2000,
"getLastErrorModes" : {

},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("59082229517dd35bb9fd0d2a")
}
}


其中,settings中的选项解释如下:

chainingAllowed:是否允许级联复制

heartbeatIntervalMillis:心跳检测时间,默认是2s

heartbeatTimeoutSecs:心跳检测失效时间,默认为10s,即如果在10s内没有收到节点的心跳信息,则判断节点不可达(HostUnreachable),对primary和Secondary均适用。

 

日志输出信息如下:

# vim /var/log/mongodb/27017.log


 View
Code

 

添加节点

myapp:PRIMARY> rs.add("node3:27018")
{ "ok" : 1 }


 

27017端口实例的日志信息如下:



2017-05-02T15:54:44.737+0800 I COMMAND  [conn1] command local.system.replset appName: "MongoDB Shell" command: count { count: "system.replset", query: {}, fields: {} } planSummary: COUNT keysExamined:0 docsExamined:0 numYields:0 reslen:29 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 135ms
2017-05-02T15:54:44.765+0800 I REPL     [conn1] replSetReconfig admin command received from client
2017-05-02T15:54:44.808+0800 I REPL     [conn1] replSetReconfig config object with 2 members parses ok
2017-05-02T15:54:44.928+0800 I ASIO     [NetworkInterfaceASIO-Replication-0] Connecting to node3:27018
2017-05-02T15:54:44.979+0800 I ASIO     [NetworkInterfaceASIO-Replication-0] Successfully connected to node3:27018
2017-05-02T15:54:44.994+0800 I NETWORK  [thread1] connection accepted from 192.168.244.30:38291 #3 (3 connections now open)
2017-05-02T15:54:45.007+0800 I NETWORK  [conn3] received client metadata from 192.168.244.30:38291 conn3: { driver: { name: "NetworkInterfaceASIO-Replication", version: "3.4.2" }, os: { type: "Linux", name: "Red Hat Enterprise Linux Server release 6.7 (Santiago)", architecture: "x86_64", version: "Kernel 2.6.32-573.el6.x86_64" } }
2017-05-02T15:54:45.009+0800 I NETWORK  [thread1] connection accepted from 192.168.244.30:38292 #4 (4 connections now open)
2017-05-02T15:54:45.010+0800 I -        [conn4] end connection 192.168.244.30:38292 (4 connections now open)
2017-05-02T15:54:45.105+0800 I REPL     [ReplicationExecutor] New replica set config in use: { _id: "myapp", version: 2, protocolVersion: 1, members: [ { _id: 0, host: "node3:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 1, host: "node3:27018", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, catchUpTimeoutMillis: 2000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId('59082229517dd35bb9fd0d2a') } }
2017-05-02T15:54:45.105+0800 I REPL     [ReplicationExecutor] This node is node3:27017 in the config
2017-05-02T15:54:45.155+0800 I REPL     [ReplicationExecutor] Member node3:27018 is now in state STARTUP
2017-05-02T15:54:45.155+0800 I COMMAND  [conn1] command local.system.replset appName: "MongoDB Shell" command: replSetReconfig { replSetReconfig: { _id: "myapp", version: 2, protocolVersion: 1, members: [ { _id: 0, host: "node3:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 1.0, host: "node3:27018" } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, catchUpTimeoutMillis: 2000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId('59082229517dd35bb9fd0d2a') } } } numYields:0 reslen:22 locks:{ Global: { acquireCount: { r: 3, w: 2, W: 1 } }, Database: { acquireCount: { w: 1, W: 1 } }, Metadata: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } protocol:op_command 403ms
2017-05-02T15:54:47.010+0800 I NETWORK  [thread1] connection accepted from 192.168.244.30:38293 #5 (4 connections now open)
2017-05-02T15:54:47.011+0800 I -        [conn5] end connection 192.168.244.30:38293 (4 connections now open)
2017-05-02T15:54:47.940+0800 I NETWORK  [thread1] connection accepted from 192.168.244.30:38294 #6 (4 connections now open)
2017-05-02T15:54:47.941+0800 I NETWORK  [conn6] received client metadata from 192.168.244.30:38294 conn6: { driver: { name: "NetworkInterfaceASIO-RS", version: "3.4.2" }, os: { type: "Linux", name: "Red Hat Enterprise Linux Server release 6.7 (Santiago)", architecture: "x86_64", version: "Kernel 2.6.32-573.el6.x86_64" } }
2017-05-02T15:54:48.010+0800 I NETWORK  [thread1] connection accepted from 192.168.244.30:38295 #7 (5 connections now open)
2017-05-02T15:54:48.011+0800 I NETWORK  [conn7] received client metadata from 192.168.244.30:38295 conn7: { driver: { name: "NetworkInterfaceASIO-RS", version: "3.4.2" }, os: { type: "Linux", name: "Red Hat Enterprise Linux Server release 6.7 (Santiago)", architecture: "x86_64", version: "Kernel 2.6.32-573.el6.x86_64" } }
2017-05-02T15:54:49.159+0800 I REPL     [ReplicationExecutor] Member node3:27018 is now in state SECONDARY
2017-05-02T15:54:49.160+0800 I -        [conn6] end connection 192.168.244.30:38294 (5 connections now open)
2017-05-02T15:55:03.401+0800 I NETWORK  [thread1] connection accepted from 192.168.244.30:38296 #8 (5 connections now open)
2017-05-02T15:55:03.403+0800 I NETWORK  [conn8] received client metadata from 192.168.244.30:38296 conn8: { driver: { name: "NetworkInterfaceASIO-RS", version: "3.4.2" }, os: { type: "Linux", name: "Red Hat Enterprise Linux Server release 6.7 (Santiago)", architecture: "x86_64", version: "Kernel 2.6.32-573.el6.x86_64" } }


 

27018端口实例的日志信息如下:


 View
Code

 

添加仲裁节点

myapp:PRIMARY> rs.addArb("node3:27019")
{ "ok" : 1 }


 

27017端口实例的日志信息如下:


 View
Code

 

27019端口实例的日志信息如下:


 View
Code

 

检查复制集的状态

myapp:PRIMARY> rs.status()
{
"set" : "myapp",
"date" : ISODate("2017-05-02T08:10:59.174Z"),
"myState" : 1,
"term" : NumberLong(1),
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1493712649, 1),
"t" : NumberLong(1)
},
"appliedOpTime" : {
"ts" : Timestamp(1493712649, 1),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1493712649, 1),
"t" : NumberLong(1)
}
},
"members" : [
{
"_id" : 0,
"name" : "node3:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 7537,
"optime" : {
"ts" : Timestamp(1493712649, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2017-05-02T08:10:49Z"),
"electionTime" : Timestamp(1493705257, 2),
"electionDate" : ISODate("2017-05-02T06:07:37Z"),
"configVersion" : 3,
"self" : true
},
{
"_id" : 1,
"name" : "node3:27018",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 974,
"optime" : {
"ts" : Timestamp(1493712649, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1493712649, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2017-05-02T08:10:49Z"),
"optimeDurableDate" : ISODate("2017-05-02T08:10:49Z"),
"lastHeartbeat" : ISODate("2017-05-02T08:10:57.606Z"),
"lastHeartbeatRecv" : ISODate("2017-05-02T08:10:58.224Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "node3:27017",
"configVersion" : 3
},
{
"_id" : 2,
"name" : "node3:27019",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 240,
"lastHeartbeat" : ISODate("2017-05-02T08:10:57.607Z"),
"lastHeartbeatRecv" : ISODate("2017-05-02T08:10:54.391Z"),
"pingMs" : NumberLong(0),
"configVersion" : 3
}
],
"ok" : 1
}


 

副本集也可通过配置文件的方式进行创建

> cfg={
...     "_id":"myapp",
...     "members":[
...         {"_id":0,"host":"node3:27017"},
...         {"_id":1,"host":"node3:27018"},
...         {"_id":2,"host":"node3:27019","arbiterOnly" : true}
... ]}

> rs.initiate(cfg)


 

验证副本集的可用性

在primary中创建一个集合,并插入一个文档进行测试

# mongo
myapp:PRIMARY> show dbs;
admin  0.000GB
local  0.000GB
myapp:PRIMARY> use test
switched to db test
myapp:PRIMARY> db.blog.insert({"title":"My Blog Post"})
WriteResult({ "nInserted" : 1 })
myapp:PRIMARY> db.blog.find();
{ "_id" : ObjectId("59082731008c534e0763e90a"), "title" : "My Blog Post" }
myapp:PRIMARY> quit()


 

在secondary中进行验证

# mongo --port 27018
myapp:SECONDARY> use test
switched to db test
myapp:SECONDARY> db.blog.find()
Error: error: {
"ok" : 0,
"errmsg" : "not master and slaveOk=false",
"code" : 13435,
"codeName" : "NotMasterNoSlaveOk"
}
myapp:SECONDARY> rs.slaveOk()
myapp:SECONDARY> db.blog.find()
{ "_id" : ObjectId("59082731008c534e0763e90a"), "title" : "My Blog Post" }
myapp:SECONDARY> quit()


 

因仲裁节点实际上并不存储任何数据,所以无法通过连接仲裁节点查看刚刚插入的文档

# mongo --port 27019
myapp:ARBITER> use test
switched to db test
myapp:ARBITER> db.blog.find();
Error: error: {
"ok" : 0,
"errmsg" : "not master and slaveOk=false",
"code" : 13435,
"codeName" : "NotMasterNoSlaveOk"
}
myapp:ARBITER> rs.slaveOk()
myapp:ARBITER> db.blog.find()
Error: error: {
"ok" : 0,
"errmsg" : "node is not in primary or recovering state",
"code" : 13436,
"codeName" : "NotMasterOrSecondary"
}
myapp:ARBITER> quit()


 

模拟primary宕掉后,副本集的自动切换

# ps -ef |grep mongodb
root       2619      1  1 13:59 ?        00:02:58 mongod --replSet myapp --dbpath /data/27018 --port 27018 --logpath /var/log/mongodb
/27018.log --forkroot       2643      1  1 13:59 ?        00:02:38 mongod --replSet myapp --dbpath /data/27019 --port 27019 --logpath /var/log/mongodb
/27019.log --forkroot       2739      1  1 14:05 ?        00:03:12 mongod --replSet myapp --dbpath /data/27017 --port 27017 --logpath /var/log/mongodb
/27017.log --forkroot       3009   2037  0 16:08 pts/2    00:00:00 vim /var/log/mongodb/27017.log
root       3055   2884  0 16:59 pts/5    00:00:00 tailf /var/log/mongodb/27019.log
root       3071   2209  0 16:59 pts/3    00:00:00 tailf /var/log/mongodb/27018.log
root       3097   1921  0 17:00 pts/0    00:00:00 grep mongodb
# kill -9 2739


 

检查复制集的状态

在这里,连接27018端口实例

# mongo --port 27018

myapp:PRIMARY> db.isMaster()
{
"hosts" : [
"node3:27017",
"node3:27018"
],
"arbiters" : [
"node3:27019"
],
"setName" : "myapp",
"setVersion" : 3,
"ismaster" : true,
"secondary" : false,
"primary" : "node3:27018",
"me" : "node3:27018",
"electionId" : ObjectId("7fffffff0000000000000002"),
"lastWrite" : {
"opTime" : {
"ts" : Timestamp(1493716742, 1),
"t" : NumberLong(2)
},
"lastWriteDate" : ISODate("2017-05-02T09:19:02Z")
},
"maxBsonObjectSize" : 16777216,
"maxMessageSizeBytes" : 48000000,
"maxWriteBatchSize" : 1000,
"localTime" : ISODate("2017-05-02T09:19:04.870Z"),
"maxWireVersion" : 5,
"minWireVersion" : 0,
"readOnly" : false,
"ok" : 1
}


可见,primary已经切换到27018端口实例上了。

对应的,27018端口实例的日志输出信息如下:


 View
Code

从日志输出中可以看出,

在第一次探测到primary不可用时,mongodb会剔除掉不健康连接(dropping unhealthy pooled connection to node3:27017),然后继续探测,直到到达10s(heartbeatTimeoutSecs)的限制,此时进行primary的自动切换。


 View
Code

 

实际上,在27017端口实例宕掉的过程中,其它两个节点均会继续针对27017端口实例进行心跳检测

2017-05-02T17:46:08.384+0800 I ASIO     [NetworkInterfaceASIO-Replication-0] Connecting to node3:27017
2017-05-02T17:46:08.384+0800 I ASIO     [NetworkInterfaceASIO-Replication-0] Failed to connect to node3:27017 - HostUnreachable: Conn
ection refused
2017-05-02T17:46:08.384+0800 I REPL     [ReplicationExecutor] Error in heartbeat request to node3:27017; HostUnreachable: Connection
refused


 

当27017端口实例重新上线时,会自动以Secondary角色加入到副本集中

27017端口实例启动并重新加入副本集的日志信息输出如下:


 View
Code

 

参考

1. 《MongoDB实战》

2. 《MongoDB权威指南》

3.  官方文档
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: