我可以在 Django 类中使用 Scrapy 函数吗
我已经在 Django 中构建了一个网站。我需要使用网络爬行功能。所以我安装了Scrapy。正如他们的教程中所述，Scrapy 正在工作，
scrapy startproject dmoz
但它适用于具有自己文件的 diff 文件夹。
是否可以在 Django 类中使用 Scrapy 函数并将数据放入 Django 变量中？
I have build a website in Django. I need to use the web crawling features. So I installed Scrapy. Scrapy is working, as stated in their tutorial, by using
scrapy startproject dmoz
But that works on a diff folder with its own files.
Is it possible to use Scrapy functions in Django classes and put data in Django variables?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
爬行是一项漫长而耗时的任务。它不能很好地与短 HTTP 请求混合。因此，您应该使用用户界面（大概是用 Django 编写的）来触发 Scrapy 爬行，并使用共享存储来监视爬行的状态。
但您不希望在 HTTP 请求的生命周期内进行爬网。这不是 Scrapy 的用途。
Crawling is a long, time consuming task. It doesn't mix well with short HTTP requests. Therefore, you should use a user interface (presumably written in Django) to fire off Scrapy crawls, and a shared storage to monitor the state of the crawl.
But you wouldn't want to crawl during the lifetime of a HTTP requests. That's not how Scrapy is meant to be used.