确保 Python 中的页面已正确下载
我正在 Python 中使用 Mechanize 和 BeautifulSoup (BS) 编写一个基本的屏幕抓取脚本。然而，我遇到的问题是，由于某种原因，所请求的页面每次都不能正确下载。我得出这个结论是因为当使用 BS 搜索当前标签的下载页面时，我收到错误。如果我再次下载该页面，它就可以工作。
因此，我想编写一个小函数来检查页面是否已正确下载，并在必要时重新下载（我也可以通过找出问题所在来解决它，但这对我来说可能太先进了）。我的问题是如何检查页面是否已正确下载？
I am writing a basic screen scraping script using Mechanize and BeautifulSoup (BS) in Python. However, the problem I am running into is that for some reason the requested page does not download correctly every time. I am concluding this because when searching the downloaded pages using BS for present tags, I get an error. If I download the page again, it works.
Hence, I would like to write a small function that checks to see if the page has correctly downloaded and re-download if necessary (I could also solve it by figuring out what goes wrong, but that is probably too advanced for me). My question is how would I go about checking to see if the page has been downloaded correctly?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
您可以只检查您期望的标签，如果失败，请重复下载。
You can just check for a tag you expect to be there, and if it fails, repeat the download.
我认为您可以简单地搜索 html 结束标记（如果此标记位于 - 这是一个有效页面）。
I think you may simple search for html ending tag if this tag is in - this is a valid page.
最通用的解决方案是检查
结束标记是否存在。这将使您能够检测页面的截断。除此之外，您还必须更清楚地描述您的故障模式。
The most generic solution is to check that the
</html>closing tag exists. That will allow you to detect truncation of the page.Anything else, and you will have to describe your failure mode more clearly.