向大文件添加标头的最佳方法是什么

发布于 2024-10-29 20:49:07 字数 65 浏览 1 评论 0原文

如果文件大小约为 1-2GB,向现有文件添加标头(例如文件元数据)的最佳方法是什么?

在 C++ 中

What 's the best way to add a header (e.g. file metadata) to an existing file if the file size is around 1-2GB?

in C++

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

—━☆沉默づ 2024-11-05 20:49:08

我想如果你真的想要超级性能,你也可以通过一些低级文件系统操作来做到这一点。但我不建议走那条路。

我也喜欢 Sege Dundich 发布的想法,即在最初写入文件时为标题保留空间,但您可能没有该选项。

I suppose you could also do it with some low-level filesystem manipulation, if you really want super performance. But I wouldn't recommend going down that road.

I also like the idea that Sege Dundich posted, about reserving space for the header when the file is being originally written, but you may not have that option available to you.

明月松间行 2024-11-05 20:49:07

最好的方法是简单地将标头写入新文件,然后附加旧文件的内容。

C++ 不提供在现有文件开头插入的方法,因此这确实是最好的选择。只需确保从旧文件中读取足够大的块并将它们附加到新文件中即可。虽然缓冲可以缓解小块执行此操作的大部分问题,但更多函数调用仍然会导致性能下降。

这可能是最小的,但它仍然存在。对于 2G 文件,我可能会从半千兆块开始,除非内存非常宝贵。这允许更大的文件大小,而不会浪费太多内存,并且四次读/写调用不太可能成为性能问题。

但是,与所有优化一样,要测量,不要猜测。有各种低级事物可能会影响 C++ 标准文档未提及的性能(而且确实如此)。由于您的问题没有提及特定的操作系统,因此我已据此回答,但这些特定的操作系统可能会(a)做出不同的反应; (b) 提供其他可以加快速度的非标准调用。

The best way is to simply write the header to a new file, then append the contents of the old.

C++ doesn't provide a way to insert at the start of an existing file so that's really the best bet. Just make sure you read large enough chunks from the old file and append them to the new one. While buffering will alleviate most of the problems of doing this in small chunks, you'll still have the performance degradation of more function calls.

This may be minimal but it's still there. For a 2G file, I'd probably start by doing it in half-gig chunks unless memory is at a premium. This allows for larger file sizes without too much memory wastage and four read/write calls is unlikely to be a performance issue.

But, as with all optimisations, measure, don't guess. There are various low level things which can affect performance that the C++ standards document makes no mention of (and rightly so). Since your question makes no mention of a specific operating system, I've answered based on that, but those specific operating systems may both (a) react differently; and (b) provide other non-standard calls which can be made faster.

懒的傷心 2024-11-05 20:49:07

在任何语言中,唯一的方法是将标头写入新文件,将旧文件的内容复制到其中,然后重命名该文件。

The only way to do it, in any language, is to write out the header to a new file, copy the contents of the old file to it, and then rename the file.

失去的东西太少 2024-11-05 20:49:07

取决于 (1) 文件系统及其参数(例如块的大小),(2) 必须前置的标头大小,(3) 谁正在写入文件。

如果您自己生成文件 (1-2Gb),则只需在文件开头为标头保留字节即可。

Depends on (1) filesystem and its parameters (like the size of the block), (2) size of the header you have to prepend, (3) who is writing the file.

If you produce the file (1-2Gb) yourself then just reserve bytes for the header at the beginning of the file.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文