pyspider mongodb 插入结果报错
爬取过程中遇到如下问题:
[I 160115 16:07:55 scheduler:671] task done douban:365c9df4123aa5e6969ffaa68c3de8a8 http://www.douban.com/note/388051131/?from=tag_all
[I 160115 16:07:55 scheduler:730] select douban:147c1576a5d2cc8c108682832e7e5c01 http://www.douban.com/note/503075232/?from=tag_all
[I 160115 16:07:55 scheduler:730] select ctrip:0a20cae937692b0ad2bdbaad24e0e22e http://you.ctrip.com/travels/canada100029/2219937.html
[I 160115 16:07:55 tornado_fetcher:306] [200] douban:147c1576a5d2cc8c108682832e7e5c01 http://www.douban.com/note/503075232/?from=tag_all 0.24s
[I 160115 16:07:55 processor:199] process douban:147c1576a5d2cc8c108682832e7e5c01 http://www.douban.com/note/503075232/?from=tag_all -> [200] len:46121 -> result:{'category fol:0 msg:0 err:None
[I 160115 16:07:55 result_worker:33] result douban:147c1576a5d2cc8c108682832e7e5c01 http://www.douban.com/note/503075232/?from=tag_all -> {'category': '\xe7\xbe\x8e\xe9
[E 160115 16:07:55 result_worker:63] new file allocation failure
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pyspider-0.3.7-py2.6.egg/pyspider/result/result_worker.py", line 54, in run
self.on_result(task, result)
File "/usr/lib/python2.6/site-packages/pyspider-0.3.7-py2.6.egg/pyspider/result/result_worker.py", line 38, in on_result
result=result
File "/usr/lib/python2.6/site-packages/pyspider-0.3.7-py2.6.egg/pyspider/database/mongodb/resultdb.py", line 51, in save
{'taskid': taskid}, {"$set": self._stringify(obj)}, upsert=True
File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 552, in update
_check_write_command_response(results)
File "/usr/lib64/python2.6/site-packages/pymongo/helpers.py", line 209, in _check_write_command_response
raise OperationFailure(error.get("errmsg"), error.get("code"), error)
OperationFailure: new file allocation failure
实在不知道为啥插入失败了
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论