您还可以在您不希望机器人索引/关注的页面中使用 ，但请始终记住这些文件中的所有内容是自愿的，机器人可以选择不跟随，所以我建议 IP 或用户代理阻止作为更好的路线。
You could also use <meta name="robots" content="noindex, nofollow" /> in the pages you dont want to robots to index/follow, however always remember that everything in these files is voluntary and the robots can choose not to follow so I recommend ip or user agent blocking as a better route.
这适用于 Google，但不保证适用于其他蜘蛛。正如 Secretformula 建议的那样，最好的选择是在服务器端逻辑中使用 ip 或用户代理阻止
User-agent: *
Disallow: /*/settings
This will work with Google, but isn't guaranteed to work with other spiders. As secretformula suggested, your best bet is to go with ip or user agent blocking in your server side logic
发布评论
评论（2）
您还可以在您不希望机器人索引/关注的页面中使用
，但请始终记住这些文件中的所有内容是自愿的，机器人可以选择不跟随，所以我建议 IP 或用户代理阻止作为更好的路线。You could also use
<meta name="robots" content="noindex, nofollow" />in the pages you dont want to robots to index/follow, however always remember that everything in these files is voluntary and the robots can choose not to follow so I recommend ip or user agent blocking as a better route.这适用于 Google，但不保证适用于其他蜘蛛。正如 Secretformula 建议的那样，最好的选择是在服务器端逻辑中使用 ip 或用户代理阻止
This will work with Google, but isn't guaranteed to work with other spiders. As secretformula suggested, your best bet is to go with ip or user agent blocking in your server side logic