pyspider 404 没有触发 catch_status_code_error 的函数

Question

代码如下：

#!/usr/bin/env python
# -*- encoding: utf-8 -*-
# Created on 2017-09-20 10:25:07
# Project: testerror

from pyspider.libs.base_handler import *


class Handler(BaseHandler):
    crawl_config = {
    }

    @every(minutes=24 * 60)
    def on_start(self):
        self.crawl('http://httpbin.org/status/404', callback=self.index_page)

    @config(age=10 * 24 * 60 * 60)
    def index_page(self, response):
        for each in response.doc('a[href^="http"]').items():
            self.crawl(each.attr.href, callback=self.detail_page)

    @config(priority=2)
    def detail_page(self, response):
        return {
            "url": response.url,
            "title": response.doc('title').text(),
        }
    
    
    @catch_status_code_error  
    def callback(self, response):
        print("==============")
        print(response)

在调试的时候， 控制台只是抛出了错误，好像没有调用callback函数。

[E 170920 13:00:22 base_handler:203] HTTP 404: NOT FOUND
    Traceback (most recent call last):
      File "/home/messyidea/.pyenv/versions/anaconda3-4.4.0/envs/py3-venv/lib/python3.6/site-packages/pyspider/libs/base_handler.py", line 196, in run_task
        result = self._run_task(task, response)
      File "/home/messyidea/.pyenv/versions/anaconda3-4.4.0/envs/py3-venv/lib/python3.6/site-packages/pyspider/libs/base_handler.py", line 175, in _run_task
        response.raise_for_status()
      File "/home/messyidea/.pyenv/versions/anaconda3-4.4.0/envs/py3-venv/lib/python3.6/site-packages/pyspider/libs/response.py", line 184, in raise_for_status
        raise http_error
    requests.exceptions.HTTPError: HTTP 404: NOT FOUND

不知道是不是哪里有问题。

还有，我不太清楚 404 之后会把任务 schedule 到 retry 还是 failed 。
可不可以在 callback 显式地把任务标记为完成？

append

搞错用法了，应该把装饰器放到index_page头上。。

Answer 1

是的，这里的装饰器实际是一个取消捕获异常的操作。