建议:Python框架服务器/工作队列管理(不是网站)

发布于 2024-12-17 10:46:53 字数 795 浏览 3 评论 0原文

我正在寻找一些关于在中央队列管理器协调的多个“工作”PC 的实现中使用哪种 Python 框架的建议/意见。

为了完整起见,“工作”PC 将运行音频转换例程(我不需要建议,并且具有可以工作的独立代码)。

音频转换需要很长时间,我需要从一个中心位置协调任意数量的“Workers”,交给他们转换任务(例如从哪里获取源文件,或者在哪里询问作业配置) ),他们报告一些附加信息,例如转换后的音频的运行时间等。

目前,我有一个脚本,可以根据位于工作器上的源文件进行 Web 服务调用以获取转换任务的“配置”已经(我们手动将源文件复制到工作人员,并触发转换例程)。我想改变这一点,以便我们可以根据可用性来分配转换任务(“哦,你,处理这个:xxx”),在理想的情况下,也可以根据待处理的任务来分配。

Workers 有可能在转换过程中离线(但这不太可能)。

所有的worker都是基于Windows的,协调器可以是WIndows或Linux。

我(在我最初的搜索中)遇到了以下内容 - 我知道其中一些是相互依赖的:

Celery (与 RabbitMQ) 扭曲的 Django

使用框架,而不是自制框架,现在对我来说似乎更有意义。我开发此功能扩展的时间有限。

另一个考虑因素是使用与 PyQT/PySide 兼容的框架,以便我可以编写一个简单的 UI 来显示队列状态等。

我知道上面的具体细节有点模糊,我希望有人能给我指点或两个。

再次强调:我正在寻找有关进一步研究哪个 Python 框架的一般建议,用于开发服务器/工作线程“队列管理”解决方案、非 Web 活动(这就是 DJango 似乎不合适的原因)。

I am looking for some advice/opinions of which Python Framework to use in an implementation of multiple 'Worker' PCs co-ordinated from a central Queue Manager.

For completeness, the 'Worker' PCs will be running Audio Conversion routines (which I do not need advice on, and have standalone code that works).

The Audio conversion takes a long time, and I need to co-ordinate an arbitrary number of the 'Workers' from a central location, handing them conversion tasks (such as where to get the source files, or where to ask for the job configuration) with them reporting back some additional info, such as the runtime of the converted audio etc.

At present, I have a script that makes a webservice call to get the 'configuration' for a conversion task, based on source files located on the worker already (we manually copy the source files to the worker, and that triggers a conversion routine). I want to change this, so that we can distribute conversion tasks ("Oy you, process this: xxx") based on availability, and in an ideal world, based on pending tasks too.

There is a chance that Workers can go offline mid-conversion (but this is not likely).

All the workers are Windows based, the co-ordinator can be WIndows or Linux.

I have (in my initial searches) come across the following - and I know that some are cross-dependent:

Celery (with RabbitMQ)
Twisted
Django

Using a framework, rather than home-brewing, seems to make more sense to me right now. I have a limited timeframe in which to develop this functional extension.

An additional consideration would be using a Framework that is compatible with PyQT/PySide so that I can write a simple UI to display Queue status etc.

I appreciate that the specifics above are a little vague, and I hope that someone can offer me a pointer or two.

Again: I am looking for general advice on which Python framework to investigate further, for developing a Server/Worker 'Queue management' solution, for non-web activities (this is why DJango didn't seem the right fit).

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

随波逐流 2024-12-24 10:46:53

使用火焰兵怎么样?它为您提供远程对象功能,您只需要一个客户端脚本来协调工作。

How about using pyro? It gives you remote object capability and you just need a client script to coordinate the work.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文