限制JS链接被搜索引擎抓取
我想阻止谷歌跟踪我在 JS 中的链接。 我在 robots.txt 中没有找到如何做到这一点 我是不是找错地方了?
更多信息: 我看到谷歌正在抓取这些页面,尽管链接只出现在 JS 中。 我不希望他抓取的原因是该内容依赖于外部 API,我不想在 google 抓取工具上浪费我的速率限制,并且仅根据用户需求
I would like to prevent google from following links I have in JS.
I didn't find how to do that in robots.txt
Am I looking in the wrong place?
Some more information:
I'm seeing google is crawling those pages although the links only appear in JS.
The reason I don't want him to crawl is that this content depends on external API's which I don't want to waste my rate limit with them on google crawlers and only per user demand
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
直接来自谷歌 ->
http://www.google.com/support/webmasters/bin /answer.py?answer=96569
Direct from google ->
http://www.google.com/support/webmasters/bin/answer.py?answer=96569
Google 可能不会找到您在 JS 中隐藏的任何链接,但其他人可能会链接到同一位置。
重要的不是链接,而是 URL。只需在 robots.txt 中指定您不希望搜索引擎访问的 URL。您通常通过 JS 将它们暴露给浏览器这一事实是无关紧要的。
如果您确实想限制对内容的访问,那么仅仅降低可发现性可能还不够,您应该放置一个身份验证层(例如密码保护)。
Google probably won't find any links you have hidden in JS, but someone else could link to the same place.
It isn't links that matter though, it is URLs. Just specify the URLs you don't want search engines to visit in the robots.txt. The fact that you usually expose them to the browser via JS is irrelevant.
If you really want to limit access to the content, then just reducing discoverability probably isn't sufficient and you should put an authentication layer (e.g. password protection) in place.