Mongodb: Db-level Locking VS Collection-level Locking
2011-09-28 11:53
288 查看
期盼中的Mongodb 2.0已经release,但是可怜的Collection-level Locking并没有实现,具体可以查看jira: https://jira.mongodb.org/browse/SERVER-1240
先说一下为什么这么期盼Collection-level Locking能够在MongoDB中实现呢,主要还是自己实际使用MongoDB的时候出现concurrency的问题:
使用update{multi: true}去更新一个collection中的大量documents的时候,整个Mongodb都hold住了,即使这个update仅仅update的是一个collection内的documents,其他collection的query操作也会block住,slow .............
使用mapreduce去执行一个耗时的操作的时候,也会出现上面的事情, very slow ..............
关于MongoDB Concurrency Work可以查看:http://www.mongodb.org/display/DOCS/How+does+concurrency+work
Read/write Lock(问题1)
mongod uses a read/write lock for many operations. Any number of concurrent read operations are allowed, but typically only one write
operation (although some write operations yield and in the future more concurrency will be added). The write lock acquisition is greedy: a pending write lock acquisition will prevent further read lock acquisitions until fulfilled.
Operations
On Javascript
Only one thread in the mongod process executes Javascript at a time (other database
operations are often possible concurrent with this).
Group Command
The group command takes a read lock and does not allow any other threads
to execute JavaScript while it is running.
MapReduce(问题2)
While the map and reduce functions are running, other JavaScript cannot be executed on the database.
There are also several non-JavaScript operations that MapReduce performs that take locks:
It reads from a collection (read lock)
It inserts documents into the temporary collection (write lock)
It creates a final collection or merges into or replaces an existing collection (write lock)
Multicore
With read operations, it is easy for mongod 1.3+ to saturate all cores.
However, because of the read/write lock above, write operations will not yet fully utilize all cores.
如果实在忍受不了某个opertion执行的太过缓慢的时候可以查看:http://www.mongodb.org/display/DOCS/Viewing+and+Terminating+Current+Operation
View Operions in Progress
Fields:
opid - an incrementing operation number. Use with killOp().
active - if the operation is active
waitingForLock - if true, lock has been requested but not yet granted
op - the operation type (query, update, etc.)
ns - namespace for the operation (database + collection name) (Note: if the ns begins with a question mark, the operation has yielded.)
query - the query spec, if operation is a query
lockType - the type of lock the operation requires, either read or write or none. See concurrency page.
client - address of the client who requested the operation
desc - the type of connection. conn indicates a normal client connections. Other values indicate internal threads in the server.
threadId - id of the thread
numYields - the number of the times the operation has yielded (to some other operation)
Killing an In Progress Operation
Summary: 至少在使用Mongodb 2.0之前版本的时候,对于会使用write lock的操作去操作大量document的时候,一定三思而后行!是很容易hold住MongoDB的!
先说一下为什么这么期盼Collection-level Locking能够在MongoDB中实现呢,主要还是自己实际使用MongoDB的时候出现concurrency的问题:
使用update{multi: true}去更新一个collection中的大量documents的时候,整个Mongodb都hold住了,即使这个update仅仅update的是一个collection内的documents,其他collection的query操作也会block住,slow .............
使用mapreduce去执行一个耗时的操作的时候,也会出现上面的事情, very slow ..............
关于MongoDB Concurrency Work可以查看:http://www.mongodb.org/display/DOCS/How+does+concurrency+work
Read/write Lock(问题1)
mongod uses a read/write lock for many operations. Any number of concurrent read operations are allowed, but typically only one write
operation (although some write operations yield and in the future more concurrency will be added). The write lock acquisition is greedy: a pending write lock acquisition will prevent further read lock acquisitions until fulfilled.
Operations
Operation | Lock type | Notes |
---|---|---|
OP_QUERY (query ) | Acquires read lock | see also: SERVER-517 |
OP_GETMORE (get more from cursor) | Acquires read lock | |
OP_INSERT (insert ) | Acquires write lock | Inserts are normally fast and short-lived operations |
OP_DELETE (remove ) | Acquires write lock | Yields while running to allow other operations to interleave. |
OP_UPDATE (update ) | Acquires write lock | Will yield for interleave (1.5.2+) |
map/reduce | At times locked | Allows substantial concurrent operation. |
create index | See notes | Batch build acquires write lock. But a background build option is available. |
db.eval() | Acquires write lock | |
getLastError command | Non-blocking | |
ismaster command | Non-blocking | |
serverStatus command | Non-blocking |
Only one thread in the mongod process executes Javascript at a time (other database
operations are often possible concurrent with this).
Group Command
The group command takes a read lock and does not allow any other threads
to execute JavaScript while it is running.
MapReduce(问题2)
While the map and reduce functions are running, other JavaScript cannot be executed on the database.
There are also several non-JavaScript operations that MapReduce performs that take locks:
It reads from a collection (read lock)
It inserts documents into the temporary collection (write lock)
It creates a final collection or merges into or replaces an existing collection (write lock)
Multicore
With read operations, it is easy for mongod 1.3+ to saturate all cores.
However, because of the read/write lock above, write operations will not yet fully utilize all cores.
如果实在忍受不了某个opertion执行的太过缓慢的时候可以查看:http://www.mongodb.org/display/DOCS/Viewing+and+Terminating+Current+Operation
View Operions in Progress
> db.currentOp() { "inprog" : [ { "opid" : 2348516, "active" : true, "lockType" : "read", "waitingForLock" : false, "secs_running" : 0, "op" : "query", "ns" : "meta_table", "query" : { "distinct" : "meta_table_user", "query" : { }, "key" : "main_name" }, "client" : "127.0.0.1:34803", "desc" : "conn" } ] }
Fields:
opid - an incrementing operation number. Use with killOp().
active - if the operation is active
waitingForLock - if true, lock has been requested but not yet granted
op - the operation type (query, update, etc.)
ns - namespace for the operation (database + collection name) (Note: if the ns begins with a question mark, the operation has yielded.)
query - the query spec, if operation is a query
lockType - the type of lock the operation requires, either read or write or none. See concurrency page.
client - address of the client who requested the operation
desc - the type of connection. conn indicates a normal client connections. Other values indicate internal threads in the server.
threadId - id of the thread
numYields - the number of the times the operation has yielded (to some other operation)
Killing an In Progress Operation
> db.killOp(2348516/*opid*/)
Summary: 至少在使用Mongodb 2.0之前版本的时候,对于会使用write lock的操作去操作大量document的时候,一定三思而后行!是很容易hold住MongoDB的!
相关文章推荐
- MongoDB执行计划获取(db.collection.explain())
- node.js连接MongoDB数据成功之后,抛出错误,db.collection is not a function
- MongoDB常用操作一查询find方法db.collection_name.find()
- MongoDB 3.0 集合方法 db.collection.explain()
- 使用JAVA向MongoDB中插入数据时报错The method insert(DBObject[]) in the type DBCollection is not applicable
- MongoDB_理解MongoClient / DB / DBCollection / DBCursor
- Mongodb中使用db.collection.copyTo()方法遇到的坑,请慎用!
- mongodb db.collection.remove用法
- MongoDB帮助信息:db方法,collection方法
- MongoDB常用操作一查询find方法db.collection_name.find()
- liunx底下db数据库比较 SQLite vs MySQL vs PostgreSQL vs Mongodb
- MongoDB执行计划获取(db.collection.explain())
- MongoDB 2.6 兼容性改变 -- db.collection.aggregate()改变
- 120 mongodb collections vs single collection - which one is more efficient?
- mongodb cloneCollection cloneDb
- NoSQL Solution: Evaluation and Comparison: MongoDB vs Redis, Tokyo Cabinet, and Berkeley DB [CHART]
- [caffe][db_lmdb.hpp:15] Check failed: mdb_status == 0 (3 vs. 0) 系统找不到指定的路径。
- mongodb 添加用户报错TypeError:db.addUser is not a function
- mongodb cloud store db
- NoSQL解决方案比较(MongoDB vs Redis, Tokyo Cabinet, and Berkeley DB)