处理QT中的大量数据
我需要阅读一些包含大量数据的文本文件，例如约500MB中的4个文件。
每个文件都包含几行，每行都具有这种格式：
id timestamp field1 field2 field3 field4
到目前为止，我的策略是解析每个文件，并且对于每行创建一个QTREEWIDGETITEM，并具有适当数量的字段来存储该行（这是因为在程序中，我想显示一些在qtreewidget中的这些数据中），并将所有这些项目附加到Qlist中。
此QLIST存储用于程序的所有执行，以这种方式，数据始终可用，我不再需要解析文件。
我需要所有可用的数据，因为在每一刻，我需要相对于特定时间戳间隔访问数据。
但是，这种策略在资源方面似乎太广泛了，因为我看到该程序消耗了几个GB的内存，并且最终会崩溃。
如何更好地处理此类数据？
I need to read some text files that contain a huge amount of data, say 4 files each of about 500MB.
Each file contains several lines and each line has about this format:
id timestamp field1 field2 field3 field4
My strategy so far was to parse each file and for every line creating a QTreeWidgetItem with a suitable number of fields to store that line (this because during the program I want to show some of these data in a QTreeWidget) and appending all these items to a QList.
This QList is stored for all the execution of the program, in this way data are always available and I don't need to parse the files anymore.
I need all the data available because at each moment I need to access to data relative to a particular timestamp interval.
However this strategy seems too expansive in terms of resources, because I saw that the program consumes several GBs of memory and it eventually crashes.
How can I approach in a better way the handling of such data?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
您想要的称为“懒惰加载”。
有一个示例您，如何使用
qabstractiTemModel，canfetchmore（）和fetchmore（）。What you want is called 'lazy loading'.
There is an Example in the Qt documentation which shows you, how to use
QAbstractItemModel,canFetchMore()andfetchMore().