如何让C# HttpWebRequest等待查询结果
我正在尝试抓取报价引擎,但 HttpWebResponse 仅返回“请稍候”屏幕,而不是最终结果。
有没有办法让请求等待响应?状态代码不起作用,因为“请稍候”屏幕返回 200。 - 我猜它在从数据库检索结果后重定向,但该重定向在最初的 200 之后。AllowAutoRedirect
为 true,但它在以下方面没有任何区别这个案例。
你们以前见过这个吗?
I'm trying to scrape a quote engine but HttpWebResponse returns only the "please wait" screen, not the final result.
Is there a way to make the request wait for the response? Status codes won't work because the "please wait" screen returns 200. - I guess it redirects after it retrieved the results from the database but that redirect comes after the initial 200.
AllowAutoRedirect is true but it doesn't make any difference in this case.
Has any of you seen this before?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
该网站可能使用 Javascript 或 元刷新 标记来加载最终目标页面。 HttpWebRequest 不直接支持这些。您已获得“请稍候”屏幕的源代码 - 阅读它以了解他们如何将您的浏览器重定向到最终结果。您可能可以从“请稍候”屏幕的内容中解析出一个 URL。获得该 URL 后,即可加载它。
The site is probably using either Javascript or a meta refresh tag to load the ultimate destination page. Neither of those are directly supported by HttpWebRequest. You've got the source to the "please wait" screen - read it to see exactly how they are redirecting your browser to the end result. There may be a URL you can parse out of the content for the "please wait" screen. Once you have that URL, you can just load it.
我的猜测是该网站可以告诉您正在抓取并告诉您离开。
My guess is the site can tell you are scraping and is telling you to bug off.