首页 新闻 搜索 专区 学院

python3 运行scrapy1.8错误

1
[待解决问题]


2020
-03-14 13:26:51 [twisted] CRITICAL: Unhandled Error Traceback (most recent call last): File "/usr/local/lib64/python3.6/site-packages/scrapy/commands/runspider.py", line 88, in run self.crawler_process.start() File "/usr/local/lib64/python3.6/site-packages/scrapy/crawler.py", line 309, in start reactor.run(installSignalHandlers=False) # blocking call File "/usr/local/lib64/python3.6/site-packages/twisted/internet/base.py", line 1283, in run self.mainLoop() File "/usr/local/lib64/python3.6/site-packages/twisted/internet/base.py", line 1292, in mainLoop self.runUntilCurrent() --- <exception caught here> --- File "/usr/local/lib64/python3.6/site-packages/twisted/internet/base.py", line 913, in runUntilCurrent call.func(*call.args, **call.kw) File "/usr/local/lib64/python3.6/site-packages/scrapy/utils/reactor.py", line 41, in __call__ return self._func(*self._a, **self._kw) File "/usr/local/lib64/python3.6/site-packages/scrapy/core/engine.py", line 122, in _next_request if not self._next_request_from_scheduler(spider): File "/usr/local/lib64/python3.6/site-packages/scrapy/core/engine.py", line 149, in _next_request_from_scheduler request = slot.scheduler.next_request() File "/usr/local/lib/python3.6/site-packages/scrapy_redis/scheduler.py", line 172, in next_request request = self.queue.pop(block_pop_timeout) File "/usr/local/lib/python3.6/site-packages/scrapy_redis/queue.py", line 117, in pop return self._decode_request(results[0]) File "/usr/local/lib/python3.6/site-packages/scrapy_redis/queue.py", line 47, in _decode_request obj = self.serializer.loads(encoded_request) File "/usr/local/lib/python3.6/site-packages/scrapy_redis/picklecompat.py", line 10, in loads return pickle.loads(s) builtins.ValueError: unsupported pickle protocol: 5

在centos7环境下运行scrapy-redis报错

and技术梦的主页 and技术梦 | 初学一级 | 园豆:15
提问于:2020-03-14 13:28
< >
分享
所有回答(1)
0

我也遇到了同样的问题,请问楼主最后解决了吗

billfqiang | 园豆:202 (菜鸟二级) | 2020-07-20 09:14
清除回答草稿
   您需要登录以后才能回答,未注册用户请先注册