Google 爬虫时间限制
有谁知道可以设置任何属性来通知 googlebot 在特定的日期或时间段（例如仅在周末）来抓取该网站？
谢谢，
does anyone know that it is possible to setup any property in order to inform googlebot to just come and crawl the site during specific day or time period (eg. during the weekend only)?
thanks,
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
您可以使用 XML 站点地图 来提示适当的抓取频率，但这只是一个提示提示，并且无法请求特定日期。
You can use an XML sitemap to give a hint about the appropriate crawling frequency, but it's only a hint, and requesting specific days is not possible.
您可以建议 googlebots 您更喜欢较慢的抓取速度（如果您的网站的抓取速度快于最低速度），但这会在 90 天内生效（请参阅 http://www.google.com/support/webmasters/bin/answer.py?hl=zh-CN&answer=48620) 。
更改 robots.txt 可能会出现问题，因为它是由 Google 缓存的，因此禁止抓取可能会导致网站无法抓取的时间远远超过预期。
Google 有不止一种机器人类型，因此您可以按照预期使用 robots.txt 来选择网站的哪些部分适合每种机器人进行抓取。 请参阅http://www.google.com/support/webmasters/bin/answer.py?answer=40360 。
You can advise googlebots that you prefer a slower crawl rate (if your site is being crawled at faster than the lowest rate) but this takes effect for 90 days (see http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=48620).
Changing the robots.txt could be problematic as that is cached by Google, so disallowing crawling could result in the site not being crawled for far longer than intended.
Google has more than one bot type so you may be able to be selective over which parts of the site are appropriate for each of them to crawl, using robots.txt as it is intended. See http://www.google.com/support/webmasters/bin/answer.py?answer=40360.
不这么认为，但googlebot确实会经常重新读取您的robots.txt，所以我想知道如果您在那时换入替代的robots.txt（例如使用脚本）是否会起作用？
Don't think so, but googlebot does re-read your robots.txt quite frequently, so I wonder if it would work if you swapped in an alternative robots.txt at those times, e.g. with a script?