pyspider运行中出现 HTTPError: HTTP 429: Too Many Requests

Question

如下所示：

[E 170706 05:41:50 base_handler:203] HTTP 429: Too Many Requests
    Traceback (most recent call last):
      File "/usr/local/lib/python2.7/dist-packages/pyspider/libs/base_handler.py", line 196, in run_task
        result = self._run_task(task, response)
      File "/usr/local/lib/python2.7/dist-packages/pyspider/libs/base_handler.py", line 175, in _run_task
        response.raise_for_status()
      File "/usr/local/lib/python2.7/dist-packages/pyspider/libs/response.py", line 184, in raise_for_status
        raise http_error
    HTTPError: HTTP 429: Too Many Requests

我已经爬取了23万的数据但是我觉得不只这么点，看了日志出现这样的错误

rate/burst 设置的是50/10

难道是访问太快了？

Answer 1

rate/burst 设置不对, 可以50/100