Googlebot 正在抓取我的网站并在我的评级系统中输入评级
我的评级系统允许匿名用户添加评级,但 Google 的抓取工具正在对事物进行评级。
如何确保 Googlebot 不会跟踪该链接?
My rating system allows anonymous users to add ratings, but Google's crawler is rating things.
How can I ensure that Googlebot won't follow the link?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
您不应接受任何修改数据的操作(投票、编辑帖子等)的 GET 请求。您的投票应通过 POST 请求完成,Googlebot 不会执行该请求。
这篇文章中的更多信息:你什么时候使用 POST,什么时候使用 GET?
You shouldn't accept a GET request for any action that modifies data (voting, editing a post, etc.). Your voting should be done via a POST request, which Googlebot won't perform.
More information in this SO post: When do you use POST and when do you use GET?
使用 robots.txt 指出机器人不应遵循的链接。例如,将以下内容放入
http://example.com/robots.txt
您可以在此处阅读有关 robots.txt 的更多信息:
http://www.robotstxt.org/
Google 和所有其他行为良好的机器人都会阅读并遵循机器人.txt。
如果您还遇到机器人不遵循这些指示的问题,您将必须编写一些逻辑来阻止机器人,或者至少减少它们的影响。例如,您可以记录在特定时间范围内从某个 IP 地址获得的投票数,并阻止高于该级别的投票。另一个解决方案可以是只允许发帖,并且还有一些 JS 逻辑(或类似的逻辑)来阻止垃圾邮件机器人,但这比 robots.txt 要做更多的工作,因此只有在出现问题时才投入时间。
您还可以通过在您的网络服务器中阻止行为不良的机器人的 IP 来完全阻止它们。如果您更喜欢阻止解决方案,可以尝试一些行为不良的机器人列表。
Use a robots.txt to point out links that bots shouldn't follow. For example, put the following in
http://example.com/robots.txt
You can read more about robots.txt here:
http://www.robotstxt.org/
Google and every other well-behaved bot will read and follow directions in robots.txt.
If you also have problems with bots not following those directions you will have to code some logic to block bots, or at least to decrease their impact. You can for example log how many votes you've got from an IP address in a certain time frame and block votes above that level. Another solution can be to only allow posts, and also have some JS logic (or similar) to block out spam bots, but that's much more work than robots.txt so only put time into it if it becomes a problem.
You can also block bad-behaving bots entirely by blocking their IPs in your web server. There are a few lists of bad-behaving bots out there you can try if you prefer the block solution.