如何使用 python urllib 在 HTTP/1.1 中保持活力
现在我正在这样做:(Python3,urllib)
url = 'someurl'
headers = '(('HOST', 'somehost'), /
('Connection', 'keep-alive'),/
('Accept-Encoding' , 'gzip,deflate'))
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor())
for h in headers:
opener.addheaders.append(x)
data = 'some logging data' #username, pw etc.
opener.open('somesite/login.php, data)
res = opener.open(someurl)
data = res.read()
... some stuff here...
res1 = opener.open(someurl2)
data = res1.read()
etc.
发生的事情是这样的;
我不断从服务器获得压缩响应,并且我保持登录状态(我正在获取一些如果未登录则不可用的内容),但我认为每个请求 opener.open 之间的连接正在断开;
我认为这是因为连接速度非常慢,而且似乎每次都有新的连接。两个问题:
a)如何测试连接是否确实处于活动状态/死亡状态
b) 如何使其在请求其他 url 之间保持活动状态?
小心 :)
For now I am doing this: (Python3, urllib)
url = 'someurl'
headers = '(('HOST', 'somehost'), /
('Connection', 'keep-alive'),/
('Accept-Encoding' , 'gzip,deflate'))
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor())
for h in headers:
opener.addheaders.append(x)
data = 'some logging data' #username, pw etc.
opener.open('somesite/login.php, data)
res = opener.open(someurl)
data = res.read()
... some stuff here...
res1 = opener.open(someurl2)
data = res1.read()
etc.
What is happening is this;
I keep getting gzipped responses from server and I stayed logged in (I am fetching some content which is not available if I were not logged in) but I think the connection is dropping between every request opener.open;
I think that because connecting is very slow and it seems like there is new connection every time. Two questions:
a)How do I test if in fact the connection is staying-alive/dying
b)How to make it stay-alive between request for other urls ?
Take care :)
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
这将是一个非常延迟的答案,但是:
您应该看到urllib3。它适用于 Python 2.x,但当您看到他们的 README 文档时您就会明白了。
是的,urllib 默认情况下不会保持连接活动,我现在正在为 Python 3 实现 urllib3,以保留在我的工具包中:)
This will be a very delayed answer, but:
You should see urllib3. It is for Python 2.x but you'll get the idea when you see their README document.
And yes, urllib by default doesn't keep connections alive, I'm now implementing urllib3 for Python 3 to be staying in my toolbag :)
如果您还不知道的话,感谢 urllib3,python-requests 提供了 keep-alive 功能。
Just if you didn't know yet, python-requests offer keep-alive feature, thanks to urllib3.