从 XML 文件导入大量数据时出现 Django MemoryError
我正在从 XML 文件导入记录，它会占用内存。我已经浏览了创建的 XML ElementTree，并删除了所有元素和子元素以清除内存，但它仍然堆积起来。
我终于找到了一个解决方案，并希望将这个解决方案提供给将来可能遇到相同问题的其他人。当我偶然发现这个问题时，我正在网上寻找答案： Django MemoryError - 如何使用大型数据库。我的 MemoryError 不是因为我的查询集的大小，因为它们没有那么大，但我想无论如何我都会尝试一下。我将所有objects.all() 更改为objects.all().iterator() 以用于对象上的任何循环。这并没有多大作用，因为我只是循环了大约三组不同的对象，其中没有一个是大组。我不情愿地在 settings.py 中将 DEBUG=True 更改为 DEBUG=False，它将内存使用量减少到几乎为零！来自 Django 文档：
“记住，当打开 DEBUG 运行时，Django 将记住它执行的每个 SQL 查询。这在调试时很有用，但它会快速消耗内存生产服务器。”
这不是生产服务器，它是我的开发服务器。那么，由于所有执行的查询都被存储，它们在哪里以及如何获取它们？如果它们对调试如此有帮助，为什么不更容易使用它们呢？
I am importing records from XML files, and it is chewing up memory. I've gone through the XML ElementTree that is created and deleted all of the elements and subelements to clear memory, but it still just piles up.
I finally found a solution, and wanted to have this in SO for others who may have the same problem in the future. I was scouring the web for answers, when I stumbled upon this: Django MemoryError - How to work with large databases. My MemoryError is not because of the size of my Queryset, because those aren't that big, but I figured I'd give it a try anyway. I changed all of my objects.all() to objects.all().iterator() for any loops over objects. That didn't do much, as I was only looping over about three different sets of objects, none of them huge sets. I reluctantly turned DEBUG=True to DEBUG=False in settings.py and it cut down the memory usage to almost nothing! From the Django Docs:
"It is also important to remember that when running with DEBUG turned on, Django will remember every SQL query it executes. This is useful when you're debugging, but it'll rapidly consume memory on a production server."
This is not a production server, it is my development server. So, since all of the queries executed are stored, where are they and how can I get them? If they are so helpful for debugging, why aren't they more readily available?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
以下是显示查询的方法：
有关更多信息，请参阅 此常见问题解答条目。
Here's how to display the queries:
For more info see this faq entry.
您还可以通过 Django Logging 记录 sql 查询，至少在 Django 1.3.x 中
https:// /docs.djangoproject.com/en/dev/topics/logging/
You can also log sql queries via Django Logging, at least in Django 1.3.x
https://docs.djangoproject.com/en/dev/topics/logging/