在输出文件中添加空间,而不必先读取整个文件
问题:如何将数据写入到文件开头的已存在文件中,而不覆盖已存在的内容,也不将整个文件读入内存? (例如前置)
信息:
我现在正在开发一个项目,该程序经常将数据转储到文件中。该文件很快就会膨胀到 3-4GB。我在一台只有 768mb 内存的计算机上运行此模拟。一遍又一遍地将所有数据拉到内存中将是一个巨大的痛苦并且浪费大量时间。模拟已经需要足够长的时间才能按原样运行。
该文件的结构使得它进行的转储数量在开头列出,只是一个简单的值,例如 6。每次程序进行新的转储时,我希望该数量增加,所以现在是 7。问题在于第 10 个、第 100 个、第 1000 个等等转储。程序将很好地输入 10,但删除下一行的第一个字母:
"9\n580,2995,2083,028\n..."
"10\n80,2995,2083,028\n... ”
显然,在这种情况下 580 和 80 之间的差异是显着的。我不能失去这些价值观。所以我需要一种方法在其中添加一点空间,以便我可以添加新数据而不会丢失数据或必须拉出整个文件然后重写它。
基本上我正在寻找的是一种前置函数。将数据添加到文件开头而不是结尾的东西。
用 Python 编程
~n
Question: How do you write data to an already existing file at the beginning of the file with out writing over what's already there and with out reading the entire file into memory? (e.g. prepend)
Info:
I'm working on a project right now where the program frequently dumps data into a file. this file will very quickly balloon up to 3-4gb. I'm running this simulation on a computer with only 768mb of ram. pulling all that data to the ram over and over will be a great pain and a huge waste of time. The simulation already takes long enough to run as it is.
The file is structured such that the number of dumps it makes is listed at the beginning with just a simple value, like 6. each time the program makes a new dump I want that to be incremented, so now it's 7. the problem lies with the 10th, 100th, 1000th, and so dump. the program will enter the 10 just fine, but remove the first letter of the next line:
"9\n580,2995,2083,028\n..."
"10\n80,2995,2083,028\n..."
obviously, the difference between 580 and 80 in this case is significant. I can't lose these values. so i need a way to add a little space in there so that I can add in this new data without losing my data or having to pull the entire file up and then rewrite it.
Basically what I'm looking for is a kind of prepend function. something to add data to the beginning of a file instead of the end.
Programmed in Python
~n
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
请参阅此问题的答案:
如何在 Python 中修改文本文件?
摘要:如果不读取文件就无法做到这一点(这是由于操作系统的工作方式,而不是Python的限制)
See the answers to this question:
How do I modify a text file in Python?
Summary: you can't do it without reading the file in (this is due to how the operating system works, rather than a Python limitation)
它没有解决您最初的问题,但这里有一些可能的解决方法:
前几个选项是前期工作少了一些,但提供了更多的灵活性。最后一个选项是解决当前问题的最简单的方法。
It's not addressing your original question, but here are some possible workarounds:
The first couple of options are a little more work up front, but provide more flexibility. The last option is the easiest solution to your current problem.
您可以很容易地创建一个新文件,输出您想要添加到该文件中的数据,然后复制现有文件的内容并将其附加到新文件中,然后重命名。
如果这是主要问题,这将避免读取整个文件。
You could quite easily create an new file, output the data you wish to prepend to that file and then copy the content of the existing file and append it to the new one, then rename.
This would prevent having to read the whole file if that is the primary issue.