Amazon EC2 的工作线程框架
是否有适用于 Python 的 Amazon EC2 通用工作框架?有些东西,你可以只说“运行这个Python任务X”,它就会自动将任务分配给服务器池中服务器上的一个工作进程,并在任务执行后返回报告?
在 Python 中描述任务(类、脚本等) - 在本例中从音频文件中提取信息
有某种控制器,您可以为其提供任务和文件(在我的例子中是通过 HTTP 交互) - 控制器知道可用于运行任务的 EC2 服务器池
控制器为任务分配一个 EC2 服务器 - 每个服务器可以包含 N 个工作进程(不是线程,毕竟这是 Python) ...)
任务在服务器上执行并将进度(0...100% 完成)报告回控制器可以将其报告给用户
Are there any generic worker frameworks for Amazon EC2 available for Python? Something, for which you could just say "run this Python task X" and it will automatically assign the task to one of worker process on the servers from a server pool and report back after the task has been executed?
Describe task in a Python (class, script, etc.) - in this case extract information from an audio file
Have some kind of controller for which you give the task and files (in my case through HTTP interaction) - the controller knowns the pool of EC2 servers it can use for running the task
The controller allocates an EC2 server for the task - each server could contain N amount of worker processes (not threads, this is Python after all...)
The task is executed on the server and the progress (0...100% done) is reported back to the controller which in turn can report it back to the user
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
您可以尝试 Elastic MapReduce
You could try Elastic MapReduce