第五部分 架构篇 第十五章 MongoDB Replica Sets 架构（动态增加删除结点）

1、Replica Set增加结点

MongoDB Replica Set不仅提供高可用性的解决方案，它也同时提供负载均衡的解决方案，增减Replica Sets结点在实际应用中非常普遍，比如当应用的读压力暴增时，3台结点的环境已经不能满足需求，那么就需要增加一些结点将压力平均分配一下，当应用的压力小时，可以减少一些结点来减少硬件资源的成本，总是是一个长期且持续的工作。

官方给我们提供了2个方案用于增加结点，一种是通过oplog来增加结点，一种是通过数据库快照(--fastsync)和oplog来增加结点，下面分别介绍：

1.1、通过oplog增加节点

配置并启动新节点，启动28010这个端口给新的节点：

[root@localhost bin]# ./mongod --replSet rs1 --keyFile=/usr/local/mongodb/key/r0 --fork --port 28010 --dbpath=/usr/local/mongodb/r0 --logpath=/usr/local/mongodb/log/r0.log --logappend
about to fork child process, waiting until server is ready for connections.
forked process: 41554
child process started successfully, parent exiting
[root@localhost bin]# ./mongo --port 28011
MongoDB shell version: 2.6.6
connecting to: 127.0.0.1:28011/test
rs1:PRIMARY> rs.add("localhost:28010");
{ "down" : [ "localhost:28010" ], "ok" : 1 }进行初始化：

rs1:PRIMARY> rs.status();
{
"set" : "rs1",
"date" : ISODate("2015-01-16T04:08:02Z"),
"myState" : 1,
"members" : [
{
"_id" : 1,
"name" : "localhost:28011",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 3695,
"optime" : Timestamp(1421381275, 1),
"optimeDate" : ISODate("2015-01-16T04:07:55Z"),
"electionTime" : Timestamp(1421379114, 1),
"electionDate" : ISODate("2015-01-16T03:31:54Z"),
"self" : true
},
{
"_id" : 2,
"name" : "localhost:28012",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 326,
"optime" : Timestamp(1421381275, 1),
"optimeDate" : ISODate("2015-01-16T04:07:55Z"),
"lastHeartbeat" : ISODate("2015-01-16T04:08:00Z"),
"lastHeartbeatRecv" : ISODate("2015-01-16T04:08:00Z"),
"pingMs" : 0,
"syncingTo" : "localhost:28011"
},
{
"_id" : 3,
"name" : "localhost:28010",
"health" : 1,
"state" : 6,
"stateStr" : "UNKNOWN",
"uptime" : 7,
"optime" : Timestamp(0, 0),
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2015-01-16T04:08:01Z"),
"lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
"pingMs" : 4,
"lastHeartbeatMessage" : "still initializing"
}
],
"ok" : 1
}初始化同步完成：
rs1:PRIMARY> rs.status();
{
"set" : "rs1",
"date" : ISODate("2015-01-16T04:08:36Z"),
"myState" : 1,
"members" : [
{
"_id" : 1,
"name" : "localhost:28011",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 3729,
"optime" : Timestamp(1421381275, 1),
"optimeDate" : ISODate("2015-01-16T04:07:55Z"),
"electionTime" : Timestamp(1421379114, 1),
"electionDate" : ISODate("2015-01-16T03:31:54Z"),
"self" : true
},
{
"_id" : 2,
"name" : "localhost:28012",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 360,
"optime" : Timestamp(1421381275, 1),
"optimeDate" : ISODate("2015-01-16T04:07:55Z"),
"lastHeartbeat" : ISODate("2015-01-16T04:08:36Z"),
"lastHeartbeatRecv" : ISODate("2015-01-16T04:08:36Z"),
"pingMs" : 0,
"syncingTo" : "localhost:28011"
},
{
"_id" : 3,
"name" : "localhost:28010",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 41,
"optime" : Timestamp(1421381275, 1),
"optimeDate" : ISODate("2015-01-16T04:07:55Z"),
"lastHeartbeat" : ISODate("2015-01-16T04:08:35Z"),
"lastHeartbeatRecv" : ISODate("2015-01-16T04:08:35Z"),
"pingMs" : 0,
"syncingTo" : "localhost:28011"
}
],
"ok" : 1
}验证数据同步：
[root@localhost bin]# ./mongo --port 28010
MongoDB shell version: 2.6.6
connecting to: 127.0.0.1:28010/test
rs1:SECONDARY> show collections
2015-01-16T12:11:06.299+0800 error: { "$err" : "not master and slaveOk=false", "code" : 13435 } at src/mongo/shell/query.js:131
rs1:SECONDARY> rs.slaveOK();
2015-01-16T12:11:12.135+0800 TypeError: Object function () { return "try rs.help()"; } has no method 'slaveOK'
rs1:SECONDARY> db.getMongo().setSlaveOk();
rs1:SECONDARY> show collections;
student
system.indexes
rs1:SECONDARY> db.student.find();
{ "_id" : ObjectId("54b883fb7bd891605d9c300f"), "name" : "zhangsan", "age" : 20 }
{ "_id" : ObjectId("54b8876aad5e04c1fe460154"), "name" : "lisi", "age" : 20 }
rs1:SECONDARY> 数据已经同步过来了。

1.2、数据库快照和oplog添加节点

通过oplog直接进行增加节点操作简单且无需人工干预过多，但oplog是capped collection采用循环的方式进行处理，所以采用oplog的方式进行增加节点，有可能导致数据不一致，因为日志中存储的信息有可能已经刷新过了，不过没有关系，我们可以通过数据库快照(--fastsync)和oplog集合的方式来增加节点，这种方式的操作流程是，先取某一个复制集成员的物理文件作为初始化数据，然后剩下的部分用oplog日志来追，最终达到数据一致性。

首先来删除之前添加的28010服务：

rs1:PRIMARY> rs.remove("localhost:28010");
2015-01-16T14:40:31.529+0800 DBClientCursor::init call() failed
2015-01-16T14:40:31.546+0800 Error: error doing query: failed at src/mongo/shell/query.js:81
2015-01-16T14:40:31.547+0800 trying reconnect to 127.0.0.1:28011 (127.0.0.1) failed
2015-01-16T14:40:31.549+0800 reconnect 127.0.0.1:28011 (127.0.0.1) ok
rs1:PRIMARY> rs.status();
{
"set" : "rs1",
"date" : ISODate("2015-01-16T06:40:36Z"),
"myState" : 1,
"members" : [
{
"_id" : 1,
"name" : "localhost:28011",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 12849,
"optime" : Timestamp(1421390431, 1),
"optimeDate" : ISODate("2015-01-16T06:40:31Z"),
"electionTime" : Timestamp(1421379114, 1),
"electionDate" : ISODate("2015-01-16T03:31:54Z"),
"self" : true
},
{
"_id" : 2,
"name" : "localhost:28012",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 5,
"optime" : Timestamp(1421390431, 1),
"optimeDate" : ISODate("2015-01-16T06:40:31Z"),
"lastHeartbeat" : ISODate("2015-01-16T06:40:35Z"),
"lastHeartbeatRecv" : ISODate("2015-01-16T06:40:34Z"),
"pingMs" : 0,
"lastHeartbeatMessage" : "syncing to: localhost:28011",
"syncingTo" : "localhost:28011"
}
],
"ok" : 1
}取某一个复制集成员的物理文件来做初始化数据
[root@localhost mongodb]# echo "this is rs1 super secret key">key/r4
[root@localhost mongodb]# chmod 600 key/r4
[root@localhost mongodb]#

[root@localhost mongodb]# scp -r r0 r3
[root@localhost mongodb]# ll
total 88
drwxr-xr-x. 2 root root  4096 Jan 16 10:19 bin
-rw-r--r--. 1 1046 1046 34520 Dec  9 07:30 GNU-AGPL-3.0
drwxr-xr-x. 2 root root  4096 Jan 16 10:27 key
drwxr-xr-x. 2 root root  4096 Jan 16 11:10 log
drwxr-xr-x. 4 root root  4096 Jan 16 12:07 r0
drwxr-xr-x. 4 root root  4096 Jan 16 11:22 r1
drwxr-xr-x. 4 root root  4096 Jan 16 11:22 r2
drwxr-xr-x. 4 root root  4096 Jan 16 14:41 r3
-rw-r--r--. 1 1046 1046  1359 Dec  9 07:30 README
-rw-r--r--. 1 1046 1046 17793 Dec  9 07:30 THIRD-PARTY-NOTICES
[root@localhost mongodb]#

在取完物理文件后，在student集中插入一条新文档，用于最后验证此更新也同步了。
[root@localhost bin]# ./mongo --port 28011
MongoDB shell version: 2.6.6
connecting to: 127.0.0.1:28011/test
rs1:PRIMARY> db.student.find();
{ "_id" : ObjectId("54b883fb7bd891605d9c300f"), "name" : "zhangsan", "age" : 20 }
{ "_id" : ObjectId("54b8876aad5e04c1fe460154"), "name" : "lisi", "age" : 20 }
rs1:PRIMARY> db.student.insert({name:"wangwu",age:30});
WriteResult({ "nInserted" : 1 })还是启动28013这个端口给新的节点：
[root@localhost bin]# ./mongod --replSet rs1 --keyFile=/usr/local/mongodb/key/r3 --fork --port 28013 --dbpath=/usr/local/mongodb/r3 --logpath=/usr/local/mongodb/log/r3.log --logappend --fastsync
about to fork child process, waiting until server is ready for connections.
forked process: 49910
child process started successfully, parent exiting
[root@localhost bin]#
添加28013节点：

[root@localhost bin]# ./mongo --port 28011
MongoDB shell version: 2.6.6
connecting to: 127.0.0.1:28011/test
rs1:PRIMARY> rs.add("localhost:28013");
{ "ok" : 1 }
rs1:PRIMARY> 验证数据同步：
[root@localhost bin]# ./mongo --port 28013
MongoDB shell version: 2.6.6
connecting to: 127.0.0.1:28013/test
rs1:SECONDARY> db.getMongo().setSlaveOk();
rs1:SECONDARY> db.student.find();
{ "_id" : ObjectId("54b883fb7bd891605d9c300f"), "name" : "zhangsan", "age" : 20 }
{ "_id" : ObjectId("54b8876aad5e04c1fe460154"), "name" : "lisi", "age" : 20 }
{ "_id" : ObjectId("54b8b2fe33612049e088be9a"), "name" : "wangwu", "age" : 30 }
rs1:SECONDARY>
至此，基于两种方式添加节点完成。

2、Relica Set减少节点

下面将节点28010从复制集当中去除掉，只需要执行rs.remove指令就可以了，具体如下：

[root@localhost bin]# ./mongo --port 28010
MongoDB shell version: 2.6.6
connecting to: 127.0.0.1:28010/test
rs1:SECONDARY> rs.status();
{
"set" : "rs1",
"date" : ISODate("2015-01-16T03:57:41Z"),
"myState" : 2,
"syncingTo" : "localhost:28011",
"members" : [
{
"_id" : 0,
"name" : "localhost:28010",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 13,
"optime" : Timestamp(1421379434, 1),
"optimeDate" : ISODate("2015-01-16T03:37:14Z"),
"infoMessage" : "syncing to: localhost:28011",
"self" : true
},
{
"_id" : 1,
"name" : "localhost:28011",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 13,
"optime" : Timestamp(1421379434, 1),
"optimeDate" : ISODate("2015-01-16T03:37:14Z"),
"lastHeartbeat" : ISODate("2015-01-16T03:57:40Z"),
"lastHeartbeatRecv" : ISODate("2015-01-16T03:57:40Z"),
"pingMs" : 0,
"electionTime" : Timestamp(1421379114, 1),
"electionDate" : ISODate("2015-01-16T03:31:54Z")
},
{
"_id" : 2,
"name" : "localhost:28012",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 13,
"optime" : Timestamp(1421379434, 1),
"optimeDate" : ISODate("2015-01-16T03:37:14Z"),
"lastHeartbeat" : ISODate("2015-01-16T03:57:40Z"),
"lastHeartbeatRecv" : ISODate("2015-01-16T03:57:40Z"),
"pingMs" : 0,
"syncingTo" : "localhost:28011"
}
],
"ok" : 1
}
rs1:SECONDARY> 将端口28010的节点删除：
rs1:SECONDARY> rs.remove("localhost:28010");
{
"ok" : 0,
"errmsg" : "replSetReconfig command must be sent to the current replica set primary."
}
rs1:SECONDARY> 上述提示在SECONDARY节点不能进行删除节点操作，必须在PRIMARY节点上进行。
[root@localhost bin]# ./mongo --port 28011
MongoDB shell version: 2.6.6
connecting to: 127.0.0.1:28011/test
rs1:PRIMARY> rs.remove("localhost:28010");
2015-01-16T12:02:37.374+0800 DBClientCursor::init call() failed
2015-01-16T12:02:37.383+0800 Error: error doing query: failed at src/mongo/shell/query.js:81
2015-01-16T12:02:37.385+0800 trying reconnect to 127.0.0.1:28011 (127.0.0.1) failed
2015-01-16T12:02:37.385+0800 reconnect 127.0.0.1:28011 (127.0.0.1) ok

rs1:PRIMARY> rs.status();
{
"set" : "rs1",
"date" : ISODate("2015-01-16T04:04:28Z"),
"myState" : 1,
"members" : [
{
"_id" : 1,
"name" : "localhost:28011",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 3481,
"optime" : Timestamp(1421380955, 1),
"optimeDate" : ISODate("2015-01-16T04:02:35Z"),
"electionTime" : Timestamp(1421379114, 1),
"electionDate" : ISODate("2015-01-16T03:31:54Z"),
"self" : true
},
{
"_id" : 2,
"name" : "localhost:28012",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 112,
"optime" : Timestamp(1421380955, 1),
"optimeDate" : ISODate("2015-01-16T04:02:35Z"),
"lastHeartbeat" : ISODate("2015-01-16T04:04:28Z"),
"lastHeartbeatRecv" : ISODate("2015-01-16T04:04:28Z"),
"pingMs" : 0,
"lastHeartbeatMessage" : "syncing to: localhost:28011",
"syncingTo" : "localhost:28011"
}
],
"ok" : 1
}
rs1:PRIMARY>

此时服务28010的已经被删除掉。