您的位置:首页 > 运维架构 > Linux

python在linux下使用多进程遇到3770问题解决方法

2017-08-16 18:09 876 查看
最近有工作要使用python启动多个脚本,用到了多进程,在windows下开发调试没有问题,但是到了linux上就出issue 3770的错了,这里记录一下问题和解决方案。

多进程代码:

from multiprocessing import Pool
import os
import time

def __long_time_task(name):
print('Run task %s (%s)...' % (name, os.getpid()))
start = time.time()

if name == 'dmp':
os.system('python DmpresponseAccount.py')
elif name == 'region':
os.system('python RegionAccount.py')
elif name == 'mix':
os.system('python MixAccount.py')

end = time.time()
print('Task %s runs %0.2f seconds.' % (name, (end - start)))

def startAllTask():
"""
启动所有数据统计任务
:return:
"""
print('Parent process %s.' % os.getpid())
process_list = ['dmp', 'region', 'mix']
# 利用进程池启动多个子进程
p = Pool(3)
for a_process in process_list:
p.apply_async(__long_time_task, args=(a_process,))
print('Waiting for all subprocesses done...')
p.close()
p.join()
print('All subprocesses done.')

if __name__ == '__main__':
startAllTask()


报错trace:

$ python StartAllTask.py
Parent process 15148.
Traceback (most recent call last):
File "StartAllTask.py", line 52, in <module>
startAllTask()
File "StartAllTask.py", line 42, in startAllTask
p = Pool(4)
File "/home/likexi/.jumbo/lib/python2.7/multiprocessing/__init__.py", line 232, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild)
File "/home/likexi/.jumbo/lib/python2.7/multiprocessing/pool.py", line 115, in __init__
self._setup_queues()
File "/home/likexi/.jumbo/lib/python2.7/multiprocessing/pool.py", line 209, in _setup_queues
from .queues import SimpleQueue
File "/home/likexi/.jumbo/lib/python2.7/multiprocessing/queues.py", line 48, in <module>
from multiprocessing.synchronize import Lock, BoundedSemaphore, Semaphore, Condition
File "/home/likexi/.jumbo/lib/python2.7/multiprocessing/synchronize.py", line 59, in <module>
" function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.


解决方案: https://segmentfault.com/q/1010000002572919

以下是回答原文:

原因:没有挂载/dev/shm;python安装时需要写入东西,才能开启sem_open
解决办法:(需要root权限;以下命令root账号执行)

1. 修改/etc/fstab 增加:tmpfs /dev/shm tmpfs defaults 0 0
2. mount /dev/shm
3. chmod 777 /dev/shm
4. 重装python (必须重装,没办法,安装时检查该设备是否存在,存在则可以使用sem_open)
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签:  python linux
相关文章推荐