在celery中动态添加/删除线程到工作池
如何从任务中向当前多处理池添加更多线程(并删除线程)(即 celeryd 运行时 CELERYD_CONCURRENCY = 10,但我想将其即时更改为 CELERYD_CONCURRENCY = 15)?
有一个名为 celery.concurrency.processes.TaskPool.Pool.grow 但我不知道如何从正在运行的任务中调用它,也不知道它是否是执行此操作的正确函数。
How do I add more threads (and remove threads) to the current multiprocessing pool, from within a task (i.e. celeryd was run with CELERYD_CONCURRENCY = 10 but I want to change it on-the-fly to CELERYD_CONCURRENCY = 15)?
There is a function called celery.concurrency.processes.TaskPool.Pool.grow but I have no idea how to call that from a running task or whether it is the correct function to do that.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
阅读源代码:
https://github.com/ask/celery/blob/master/celery/concurrency/processes/__init__.py
其中有
grow()
和 < code>shrink(),尽管后者看起来有点可疑。您需要在某处保留对池的引用,如果您只有一个池,请将其保持为全局。
警告海报:如果多处理实际上意味着运行多个单独的进程,那么当您尝试缩小或增长时,您可能已经处于子进程中,显然这是行不通的。
Read the source:
https://github.com/ask/celery/blob/master/celery/concurrency/processes/__init__.py
there's both
grow()
andshrink()
, although the latter seems a tad fishy.you'd need to keep a reference to the pool somewhere, if you have only one pool, keep it global.
caveat poster: if multiprocessing actually means running multiple separate processes, you might already be in a child process when you try to shrink or grow, and obviously that won't work.