Python 多处理-进程完成后如何释放内存？
我在使用 python 多处理库时遇到了一个奇怪的问题。
我的代码如下所示：我为每个“符号，日期”元组生成一个过程。之后我将结果结合起来。
我期望当一个进程完成“符号，日期”元组的计算时，它应该释放其内存？显然事实并非如此。我看到机器中有数十个进程（尽管我将进程池设置为大小 7）被挂起。它们不消耗CPU，也不释放内存。
在完成计算后，如何让进程释放其内存？
谢谢！
1 “暂停”是指它们在 ps 命令中的状态显示为“S+”
def do_one_symbol( symbol, all_date_strings ):
pool = Pool(processes=7)
results = [];
for date in all_date_strings:
res = pool.apply_async(work, [symbol, date])
results.append(res);
gg = mm = ss = 0;
for res in results:
g, m, s = res.get()
gg += g;
mm += m;
ss += s;
I encountered a weird problem while using python multiprocessing library.
My code is sketched below: I spawn a process for each "symbol, date" tuple. I combine the results afterwards.
I expect that when a process has done computing for a "symbol, date" tuple, it should release its memory? apparently that's not the case. I see dozens of processes (though I set the process pool to have size 7) that are suspended¹ in the machine. They consume no CPU, and they don't release the memory.
How do I let a process release its memory, after it has done its computation?
Thanks!
¹ by "suspended" I mean their status in ps command is shown as "S+"
def do_one_symbol( symbol, all_date_strings ):
pool = Pool(processes=7)
results = [];
for date in all_date_strings:
res = pool.apply_async(work, [symbol, date])
results.append(res);
gg = mm = ss = 0;
for res in results:
g, m, s = res.get()
gg += g;
mm += m;
ss += s;
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
您是否尝试使用
pool.close 来关闭池。关闭，然后等待pool.join，因为如果父进程继续运行并且不等待子进程，它们将变成 僵尸Did you try to close pool by using
pool.closeand then wait for process to finish bypool.join, because if parent process keeps on running and does not wait for child processes they will become zombies尝试在池上设置 maxtasksperchild 参数。如果不这样做，那么池会一遍又一遍地重用该进程，因此内存永远不会被释放。设置后，该进程将被允许终止，并在其位置创建一个新进程。这将有效地清理内存。
我想它是 2.7 中的新功能： http://docs.python.org/2/library /multiprocessing.html#module-multiprocessing.pool
Try setting the maxtasksperchild argument on the pool. If you don't, then the process is reusued over and over again by the pool so the memory is never released. When set, the process will be allowed to die and a new one created in it's place. That will effectively clean up the memory.
I guess it's new in 2.7: http://docs.python.org/2/library/multiprocessing.html#module-multiprocessing.pool
您可能应该在
Pool对象上调用close()，然后调用wait()。http://docs.python.org/library/multiprocessing.html#module -multiprocessing.pool
You should probably call
close()followed bywait()on yourPoolobject.http://docs.python.org/library/multiprocessing.html#module-multiprocessing.pool