当任何线程出现异常时,多处理池会挂起
我是 Python 新手,正在尝试使用 multiprocessing.pool 程序来处理文件,只要没有异常,它就可以正常工作。如果任何线程/进程出现异常,则整个程序将等待
代码的线程片段:
cp = ConfigParser.ConfigParser()
cp.read(gdbini)
for table in cp.sections():
jobs.append(table)
#print jobs
poolreturn = pool.map(worker, jobs)
pool.close()
pool.join()
失败消息:
Traceback (most recent call last):
File "/opt/cnet-python/default-2.6/lib/python2.6/threading.py", line 525, in __bootstrap_inner
self.run()
File "/opt/cnet-python/default-2.6/lib/python2.6/threading.py", line 477, in run
self.__target(*self.__args, **self.__kwargs)
File "/opt/cnet-python/default-2.6/lib/python2.6/multiprocessing/pool.py", line 259, in _handle_results
task = get()
TypeError: ('__init__() takes exactly 3 arguments (2 given)', <class 'ConfigParser.NoOptionError'>, ("No option 'inputfilename' in section: 'section-1'",))
我继续添加了一个异常处理程序来终止进程
try:
ifile=cp.get(table,'inputfilename')
except ConfigParser.NoSectionError,ConfigParser.NoOptionError:
usage("One of Parameter not found for"+ table)
terminate()
,但它仍然在等待,不确定缺少什么。
I am new to Python and trying a multiprocessing.pool program to process files, it works fine as long as there are no exceptions. If any of the thread/process gets an exception the whole program waits for the thread
snippet of the code:
cp = ConfigParser.ConfigParser()
cp.read(gdbini)
for table in cp.sections():
jobs.append(table)
#print jobs
poolreturn = pool.map(worker, jobs)
pool.close()
pool.join()
Failure Message:
Traceback (most recent call last):
File "/opt/cnet-python/default-2.6/lib/python2.6/threading.py", line 525, in __bootstrap_inner
self.run()
File "/opt/cnet-python/default-2.6/lib/python2.6/threading.py", line 477, in run
self.__target(*self.__args, **self.__kwargs)
File "/opt/cnet-python/default-2.6/lib/python2.6/multiprocessing/pool.py", line 259, in _handle_results
task = get()
TypeError: ('__init__() takes exactly 3 arguments (2 given)', <class 'ConfigParser.NoOptionError'>, ("No option 'inputfilename' in section: 'section-1'",))
I went ahead added a exception handler to terminate the process
try:
ifile=cp.get(table,'inputfilename')
except ConfigParser.NoSectionError,ConfigParser.NoOptionError:
usage("One of Parameter not found for"+ table)
terminate()
but still it waits, not sure whats missing.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
在 Python 3.2+ 中,这可以按预期工作。对于 Python 2,此错误已在 r74545 中修复,并将在 Python 2.7.3 中提供。同时,您可以使用 configparser 库,它是 3.2+ 版 configparser 的向后移植。 查看一下。
In Python 3.2+ this works as expected. For Python 2, this bug was fixed in r74545 and will be available in Python 2.7.3. In the mean time, you can use the
configparser
library which is a backport of the configparser from 3.2+. Check it out.我有同样的问题。当工作进程引发具有自定义构造函数的用户异常时,就会发生这种情况。确保您的异常(在这种情况下为 ConfigParser.NoOptionError)使用恰好两个参数初始化基本异常:
I had the same issue. It happens when a worker process raises a user exception which has a custom constructor. Make sure your exception (ConfigParser.NoOptionError in that case) initializes the base exception with exactly two arguments: