保护链接数据库免受自动抓取工具侵害的好方法是什么？
我有一个大型链接数据库，我希望防止其他人复制它们。除了强制人们在每个链接之前输入验证码之外，我还能做些什么吗？
I have a large link database, that I would want to protect against others who would want to copy them. Is there anything I can do other than force people to enter a CAPTCHA before each link?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
您可以使用 ROT13 输出链接，然后使用 javascript 将它们恢复正常。
这样，抓取工具必须支持 javascript 才能窃取您的链接，这应该会减少合格抓取工具
奖励积分的数量：用更难的东西替换 ROT13，并混淆您的“解码”javascript。
you can output the links using ROT13, and then use javascript to put them back to normal.
this way, the scrapers must support javascript in order to steal your links, which should cut down on the number of eligible scrapers
bonus points: replace ROT13 with something harder, and obfuscate your 'decode' javascript.
javascript 建议可行，但您会使那些使用屏幕阅读器等辅助技术以及没有 javascript 的人无法访问您的页面。
另一种可能的选择是生成加密随机数。该技术目前用于防止 CSRF 攻击，但也可用于确保抓取工具在访问链接之前必须从您的站点请求页面。如果您支持热链接，这种方法可能不合适，但如果您只是想确保有人首先访问您的网站，那么它可能会起作用。
另一个有点贫民窟的选择是使用推荐人。这些很容易被伪造，但它可能会阻止一些愚蠢的抓取工具。这还要求您在用户访问您的网站之前知道他们来自哪里。
您能否告知我们您是否存在热链接或者用户是否在访问受保护的链接之前访问您的网站？这样我们也许能够提供更好的建议。
The javascript suggestion could work, but you would render your page inaccessible to those using assistive technologies like screen readers as well as anyone without javascript.
Another possible option would be to generate a cryptographic nonce. This technique is currently used to protect against CSRF attacks, but could also be used to ensure that the scraper would have to request a page from your site before accessing a link. This approach may not be appropriate if you support hotlinking, but if you just want to make sure that someone went to your site first, it could work.
Another somewhat ghetto option would be use referrers. These can be easily faked, but it might prevent some of the dumber scrapers. This also requires that you know where your users came from before they hit your site.
Can you let us know if you are hotlinking or if the user comes to your site before going to the protected link? We might be able to provide better advice that way.