带有 urllib 的 Python 线程
我使用 python 来同时请求多个请求的 Web 服务。为此,我创建线程并使用 urllib(第一个版本,我使用 python 2.6)。 当我启动线程时,一切顺利,直到到达 ulllib.urlopen()。第二个线程必须等到第一个线程结束才能通过 ulllib.urlopen() 函数。由于我在从远程 Web 服务检索 Json 后做了很多工作,因此我希望第二个线程在同一时间或在第一个线程关闭其套接字之后“urlopen”。
我尝试关闭在收集返回的 JSON 后打开的套接字,但它没有任何改变。第二个线程必须等待第一个线程结束。看我用的是印刷品。
我可以理解 urllib 不是线程安全的(谷歌这没有给出明确的答案)但是为什么第二个线程必须等待第一个结束(而不仅仅是套接字进程结束)?
感谢您的帮助和提示
PS:我不使用 Python 3 来兼容我需要的模块/包
I use python to request a web service with many requests in the same time. To do so I create threads and use urllib (first version, I use python 2.6).
When I start the threads, all goes well until one reach the ulllib.urlopen(). The second thread has to wait until the first one end before passing through the ulllib.urlopen() function. As I do a lot of work after having retrieved the Json from remote web service, I wish the second thread to "urlopen" in the same time or just after the first one closes its socket.
I tried closing the socket opened just after having collected the JSON returned but it changes nothing. The second thread has to wait for the first one to be ended. To see that I use prints.
I can understand that urllib isn't thread-safe (google this doesn't give clear answers) but why does the second thread has to wait for the first-one end (and not just the socket process end) ?
Thanks for your help and hints
PS: I do not use Python 3 for compatibility with modules / packages I require
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
这听起来不是预期的行为,因为两个并行的 urllib 请求应该是可能的。您确定您的远程服务器可以处理两个并行请求(例如,它不处于单线程调试模式)?
无论如何:线程并不是使用 Python 进行并行编程的首选方法。使用进程或异步,尤其是在服务器端(您没有提到用例或您的平台也可能有错误)。
我在使用 Spawning 和 Eventlet 处理和转换 JSON/XML 方面拥有非常好的经验,它们将 Python 套接字代码修补为异步。
http://pypi.python.org/pypi/Spawning/
http://eventlet.net/
This does not sounds intended behavior as two parallel urllib request should be possible. Are you sure your remote server can handle two paraller requests (e.g. it is not in debug mode with a single thread)?
Any case: threading is not a preferred approach for parallel programming with Python. Either use processes or async, especially on the server side (you didn't mention the use case or your platform which may also be buggy).
I have had very good experiences processing and transforming JSON/XML with Spawning and Eventlets which patch Python socket code to be asynchronous.
http://pypi.python.org/pypi/Spawning/
http://eventlet.net/