终止响应缓慢的 HttpWebRequest
有时,在爬行时,我会得到非常慢的响应。
连接不慢,所以超时不起作用。其主机速度为 3-4 字节/秒。
如果请求时间超过 10 秒(例如超时),是否有办法终止该请求?
Occasionally while crawling, I get a VERY slow response.
It is not slow to connect, so timeout doesn't work. Its the host speed going 3-4 Bytes/second.
Is there a way to kill this request if it takes more than 10 seconds, like a timeout?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
如果您的 HttpWebRequest 在线程中,您应该能够通过在线程本身上调用 Abort 来终止请求...我知道这可能是“杀伤力过大”...也许其他人有更好的主意...
If your HttpWebRequest is in a thread you schould be able to kill the request by calling Abort on the thread itself... I know this is perhaps "overkill"... perhaps someone else has better idea...
您想要做的是在其自己的线程上启动爬网进程,该线程在指定时间后超时。看一下 HttpWebRequest 类上的 BeginGetResponse(),它启动一个线程并在完成时调用指定的函数。您可以为此方法添加超时机制,如下页的示例部分所示: http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.begingetresponse(v=VS.71).aspx。
抱歉,我没有任何方便的代码来展示这是如何完成的,但该页面上的示例看起来很完整。
What you want to do is start your crawl process on its own thread that times out after a specified time. Have a look at BeginGetResponse() on the HttpWebRequest class, which starts a thread and calls a specified function when it completes. You can add a timeout mechanism to this method, as shown under the Example section on the following page: http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.begingetresponse(v=VS.71).aspx.
Sorry, I don't have any code handy to show how this is done, but the example on that page appears complete.