需要帮助理解' scrapy'错误信息
最近,我已经完成了一项更新使用网络刮擦程序的网站的任务,但是,我是新的。该程序的先前创建者给我的说明是运行main_executor.py,网站应该毫无问题地运行。但是,事实并非如此,我收到了我不确定的一堆错误消息。我希望有更多经验的人能够看待它,并向我解释这个问题。
错误消息:
Python 3.10.5 (tags/v3.10.5:f377153, Jun 6 2022, 16:14:13) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license()" for more information.
= RESTART: C:\Users\khoih\OneDrive\Desktop\RA Files\CyberSite\research\research\main_executor.py
Traceback (most recent call last):
File "C:\Users\khoih\OneDrive\Desktop\RA Files\CyberSite\research\research\main_executor.py", line 4, in <module>
setting = get_project_settings()
File "C:\Users\khoih\AppData\Local\Programs\Python\Python310\lib\site-packages\scrapy\utils\project.py", line 68, in get_project_settings
settings.setmodule(settings_module_path, priority='project')
File "C:\Users\khoih\AppData\Local\Programs\Python\Python310\lib\site-packages\scrapy\settings\__init__.py", line 287, in setmodule
module = import_module(module)
File "C:\Users\khoih\AppData\Local\Programs\Python\Python310\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'research'
任何帮助都将在APR上进行,如果需要,我可以提供代码。
main_executor.py代码:
from scrapy.utils.project import get_project_settings
from scrapy.crawler import CrawlerProcess
setting = get_project_settings()
process = CrawlerProcess(setting)
for spider_name in process.spiders.list():
print ("Running spider %s" % (spider_name))
process.crawl(spider_name,query="dvh") #query dvh is custom argument used in your scrapy
process.start()
I've recently gotten a task to update a website that utilizes a web scraping program, however, I am fairly new to it. The instructions given to me by the prior creator of the program were to run the main_executor.py and the website should run with no problem. However, that wasn't the case I am getting a bunch of error messages that I'm unsure of. I was hoping someone with more experience could look at it and explain to me the problem.
Error message:
Python 3.10.5 (tags/v3.10.5:f377153, Jun 6 2022, 16:14:13) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license()" for more information.
= RESTART: C:\Users\khoih\OneDrive\Desktop\RA Files\CyberSite\research\research\main_executor.py
Traceback (most recent call last):
File "C:\Users\khoih\OneDrive\Desktop\RA Files\CyberSite\research\research\main_executor.py", line 4, in <module>
setting = get_project_settings()
File "C:\Users\khoih\AppData\Local\Programs\Python\Python310\lib\site-packages\scrapy\utils\project.py", line 68, in get_project_settings
settings.setmodule(settings_module_path, priority='project')
File "C:\Users\khoih\AppData\Local\Programs\Python\Python310\lib\site-packages\scrapy\settings\__init__.py", line 287, in setmodule
module = import_module(module)
File "C:\Users\khoih\AppData\Local\Programs\Python\Python310\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'research'
Any help would be aprrcieated and I can provide the code if needed.
The main_executor.py code:
from scrapy.utils.project import get_project_settings
from scrapy.crawler import CrawlerProcess
setting = get_project_settings()
process = CrawlerProcess(setting)
for spider_name in process.spiders.list():
print ("Running spider %s" % (spider_name))
process.crawl(spider_name,query="dvh") #query dvh is custom argument used in your scrapy
process.start()
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论