Twisted 是构建多线程服务器的不错选择吗?
我需要从数百个 pop3 电子邮件帐户中提取数据,并且我想构建一个强大的服务器来执行此操作。
对于此类项目来说,Twisted 是一个不错的选择吗?
现在,一个简单的原型是从单个 pop3 帐户中提取,然后从多个帐户中提取,但这将是一个序列化过程。
我想创建一个具有多个线程的服务器,以便它可以同时执行任务。
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(4)
Twisted 是一个用 Python 编写的事件驱动的网络框架。它在很大程度上建立在异步和非阻塞功能的基础上,最适合开发利用这些功能的网络应用程序。它为无法提供异步非阻塞 I/O 的用例提供线程支持。这是基于大多数时间都花在等待网络 I/O 操作上的事实。
利用这一点的两种模型是线程模型,您可以在其中创建多个线程,每个线程完成一个任务,或者使用非阻塞 I/O 通过交错多个任务在单个进程中完成多个任务的单个进程。 Twisted非常适合第二种模式。
非阻塞模型
您可以使用 Twisted 开发非常强大的服务器,并且它支持 POP3 / IMAP。
有一个如何构建 带有twisted的pop3客户端。
Twisted is an event-driven networking framework written in Python. It builds heavily on asynchronous and non-blocking features and is best conceived to develop networking applications that utilizes these. It has thread support for use cases where you can not provide for asynchronous non-blocking I/O. This is based on the fact that most of time is spent waiting in network I/O operations.
The two model that exploits this is threading model where you create multiple threads, each accomplishing a single task or a single process that uses non-blocking I/O to accomplish multiple task in a single process by interleaving multiple tasks. Twisted is very suitable for the second model.
Non-Blocking model
You can develop a very robust server with Twisted and it has POP3 / IMAP support.
There is an example of how to build pop3 client with twisted.
考虑到您的大部分 POP3 活动都是网络 I/O,这就是 Twisted 的优势所在。您实际上并不是在线程化,而是执行基于事件的异步套接字操作,这是 Twisted 的最高荣耀。
所以,是的,Twisted 对于此类项目来说是一个不错的选择。它可以同样出色地执行客户端和服务器操作,并且启动一个新的异步 TCP 客户端几乎是微不足道的,并且它已经有一个 POP3 TCP 客户端 默认可用。
Considering that the majority of your POP3 activity is going to be network I/O, this is where Twisted excels. You're not really threading so much as performing event-based asynchronous socket operations, which is the crowning glory of Twisted.
So, yes, Twisted would be a good choice for this type of project. It can do client and server operations equally well and it is almost trivial to spin up a new async TCP client and it already has a POP3 TCP Client available by default.
对于服务器来说这是一个不错的选择,但根据您的描述,您实际上正在寻找多线程 POP 客户端。
Twisted 是为了响应传入请求等事件而设计的,您需要发送请求,所以在这种情况下,我担心 Twisted 的价值有限。
It is a good choice for a server but from your description you are acutally looking for a multithreaded POP client.
Twisted is made for reacting to events like incoming requests, you need to send requests, so in this case I fear twisted to be of limited value.
需要注意的是,虽然twisted非常强大,但我发现使用文档中提供的代码示例旋转一百个线程会导致竞争条件和死锁。我的建议是尝试扭曲,但如果扭曲变得难以管理,则让 stdlib 多线程模块伺机而动。我使用上述库的生产者消费者模型取得了巨大成功。
A word of caution with twisted, while twisted is very robust I've found that spinning up a hundred threads using the code examples available in documentation is a recipe for race conditions and deadlocks. My suggestion is try twisted but have the stdlib multithreading module waiting in the wings if twisted becomes unmanageable. I have had good success with a producer consumer model using the aforementioned library.