Python多处理在Pool关闭之前执行一个函数
我正在使用 python 的 multiprocessing 模块来并行化和处理我的数据。在关闭此进程池之前，我想为生成的每个子进程执行一个函数。我怎样才能实现这个目标？
例如，下面是示例代码：
from multiprocessing import Pool
def publish_metadata():
print(str(metadata))
metadata = []
def process_row(row={}):
global metadata
metadata.append("something")
# process the data
if __name__ == "__main__":
pl = [////] # list to process data, each row is a dict
pool = Pool(processes=10)
pool.map(process_row, pl)
pool.close()
pool.join()
我想在合并和关闭进程池（pool.close）之前运行此函数publish_metadata，它将包含子进程特定的元数据。我怎样才能做到这一点？
I'm using python's multiprocessing module for parallelising and processing my data. Before I close this process pool, I want to execute a function for each of the sub-processes that are spawned. How can I achieve this?
eg, below is a sample code:
from multiprocessing import Pool
def publish_metadata():
print(str(metadata))
metadata = []
def process_row(row={}):
global metadata
metadata.append("something")
# process the data
if __name__ == "__main__":
pl = [////] # list to process data, each row is a dict
pool = Pool(processes=10)
pool.map(process_row, pl)
pool.close()
pool.join()
I want to run this function publish_metadata which will contain sub-process specific metadata, before I merge and close process pool (pool.close). How can I achieve that?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论