在celery中动态添加/删除线程到工作池

发布于 2024-11-28 00:13:26 字数 392 浏览 1 评论 0原文

如何从任务中向当前多处理池添加更多线程(并删除线程)(即 celeryd 运行时 CELERYD_CONCURRENCY = 10,但我想将其即时更改为 CELERYD_CONCURRENCY = 15)?

有一个名为 celery.concurrency.processes.TaskPool.Pool.grow 但我不知道如何从正在运行的任务中调用它,也不知道它是否是执行此操作的正确函数。

How do I add more threads (and remove threads) to the current multiprocessing pool, from within a task (i.e. celeryd was run with CELERYD_CONCURRENCY = 10 but I want to change it on-the-fly to CELERYD_CONCURRENCY = 15)?

There is a function called celery.concurrency.processes.TaskPool.Pool.grow but I have no idea how to call that from a running task or whether it is the correct function to do that.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

流年已逝 2024-12-05 00:13:26

阅读源代码:

https://github.com/ask/celery/blob/master/celery/concurrency/processes/__init__.py

其中有 grow() 和 < code>shr​​ink(),尽管后者看起来有点可疑。

您需要在某处保留对池的引用,如果您只有一个池,请将其保持为全局。

警告海报:如果多处理实际上意味着运行多个单独的进程,那么当您尝试缩小或增长时,您可能已经处于子进程中,显然这是行不通的。

Read the source:

https://github.com/ask/celery/blob/master/celery/concurrency/processes/__init__.py

there's both grow() and shrink(), although the latter seems a tad fishy.

you'd need to keep a reference to the pool somewhere, if you have only one pool, keep it global.

caveat poster: if multiprocessing actually means running multiple separate processes, you might already be in a child process when you try to shrink or grow, and obviously that won't work.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文