对 Apache 施加访问限制以防止抓取?
问题在于内容网站被严重抓取以至于破坏了服务器。
是否有一种简单的方法可以将 IP 的访问限制为一次或每天固定数量的请求? (每天 10 页或......每 2 分钟 10 页)
理想情况下,我会为搜索引擎保留一个通配符列表,并禁止其他人过快或过多地访问内容。
谢谢你们!
The problem is of a content website that is being scraped so badly that it breaks the server.
Is there an easy method of limiting access for IPs to a fixed number of requests at a time OR per day ? ( 10 pages / day or.... 10 pages every 2 minutes )
Ideally, I would keep a wildcard list for search engines and disallow everybody else from accessing content too fast or too much.
Thanks guys!
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
解决这个问题的一种方法是使用 IPTABLES(仅限 Linux)来防止单个 IP 启动超过指定数量的连接。这是反复试验,因为您需要正确计算它,但总的来说,这应该可以防止攻击者的连接率
希望它有所帮助
One way around this, would be using IPTABLES (linux only) to prevent that individual IPs start more than a specified number of connections. It's trial and error, as you need to calculate it right, but in an overall, that should prevent the attacker's connection rate
Hope it helps
我更喜欢在系统级别使用 iptables 执行此操作...
但如果您正在寻找基于 Apache 的解决方案,一个想法可能是使用
mod_security
。SecGuardianLog< /code>
配置指令看起来特别有趣,在您的情况下(引用):
I would rather prefer doing that at the system level, using iptables...
But if you're looking for a solution based on Apache, an idea might be to use
mod_security
.The
SecGuardianLog
configuration directive looks especially interesting, in your case (quoting) :您可以安装 mod_bandwidth 和 mod_limitipconn 等模块来限制带宽使用(全局和每个连接)。
查看 http://mansurovs.com/tech/apache-bandwidth-throtdling 了解更多信息信息。
You can install modules such as mod_bandwidth and mod_limitipconn to limit bandwidth usage (globally and per connection).
Check http://mansurovs.com/tech/apache-bandwidth-throttling for more info.