pyspider调试的时候没有任何问题,点run就报编码问题

Question

调试的时候没有任何问题,点run就报编码问题。同样2个采集就这一个老报错,另外一个完全没问题

taskid

d7221a2be620c4ef60e874a1d93e79d1

lastcrawltime

1499144004.75187 (20 minutes ago)

updatetime

1499144004.7518892 (20 minutes ago)

exetime

1499144014.7518687 (20 minutes ago)

track.fetch 1.32ms

{
  "content": "",
  "encoding": null,
  "error": "'ascii' codec can't encode character '\\uff09' in position 94: ordinal not in range(128)",
  "headers": {},
  "ok": false,
  "redirect_url": null,
  "status_code": 599,
  "time": 0.0013222694396972656
}

track.process 0.83ms

'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)
 = self.gen.throw(*exc_info)
      File "/root/workspaces/pyspider3/lib/python3.5/site-packages/pyspider/fetcher/tornado_fetcher.py", line 378, in http_fetch
        response = yield gen.maybe_future(self.http_client.fetch(request))
      File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/gen.py", line 1055, in run
        value = future.result()
      File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/concurrent.py", line 238, in result
        raise_exc_info(self._exc_info)
      File "<string>", line 4, in raise_exc_info
      File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py", line 214, in _process_queue
        curl.info["headers"])
      File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py", line 306, in _curl_setup_request
        for k, v in request.headers.get_all()])
    Exception: 'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)

{
  "exception": "'ascii' codec can't encode character '\\uff09' in position 94: ordinal not in range(128)",
  "follows": 0,
  "logs": " = self.gen.throw(*exc_info)\n      File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/pyspider/fetcher/tornado_fetcher.py\", line 378, in http_fetch\n        response = yield gen.maybe_future(self.http_client.fetch(request))\n      File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/gen.py\", line 1055, in run\n        value = future.result()\n      File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/concurrent.py\", line 238, in result\n        raise_exc_info(self._exc_info)\n      File \"<string>\", line 4, in raise_exc_info\n      File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py\", line 214, in _process_queue\n        curl.info[\"headers\"])\n      File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py\", line 306, in _curl_setup_request\n        for k, v in request.headers.get_all()])\n    Exception: 'ascii' codec can't encode character '\\uff09' in position 94: ordinal not in range(128)\n",
  "ok": false,
  "result": null,
  "time": 0.0008292198181152344
}

schedule

{
  "age": 10,
  "exetime": 1499144014.7518687,
  "retried": 3
}

process

{
  "callback": "index_page"
}

fetch

{}

Answer 1

headers设置有问题,我删掉直接秒好.

Answer 2

#!/usr/bin/env python
# -*- encoding: utf-8 -*-

看看你的代码前两行是这个吧