mmap() 与 read()
我正在用 C 编写一个批量 ID3 标签编辑器。ID3 标签通常位于 mp3 编码文件的开头，尽管较旧的（版本 1）标签位于末尾。该应用程序旨在从命令行接受目录和框架 ID 列表，然后递归目录结构更新它找到的所有 ID3 标签。用户还可以选择删除所有旧的（版本 1）标签。另一种选择是仅显示当前标签，而不执行更新。该目录可能包含 2 个文件或 200 万个文件。如果用户打算更新文件，我计划将整个文件加载到内存中，执行更新，然后保存它（文件也可能被重命名）。然而，如果用户只想打印当前的 ID3 标签，那么加载整个文件似乎过多。毕竟文件可能有 200mb。
我已经通读了这篇文章，它很有洞察力 - mmap() 与阅读块
所以我的问题是，最有效的方法是什么 - read()、mmap() 或某种组合？欢迎设计想法。
编辑：据我了解， mmap 本质上将文件加载到内存中委托给虚拟内存子系统。在我看来，VMM 在大多数系统上都会进行高度优化，因为它对系统性能至关重要。
I'm writing a bulk ID3 tag editor in C. ID3 tags are usually at the beginning of an mp3 encoded file, although older (version 1) tags are at the end. The app is designed to accept a directory and frame ID list from the command line, then recurse the directory structure updating all the ID3 tags it finds. The user may additionally choose to remove all older (version 1) tags. Another option is to simply display the current tags, without performing an update. The directory might contain 2 files or 2 million. If the user means to update the files, I was planning to load the entire file into memory, perform the updates, then save it (the file may be renamed as well). However, if the user only means to print the current ID3 tags, then loading the entire file seems excessive. After all the file could be 200mb.
I've read through this thread, which was insightful - mmap() vs. reading blocks
So my question is, what the most efficient way to go about this -- read(), mmap() or some combination? Design ideas welcome.
Edit: It's my understanding that mmap essentially delegates loading a file into memory, to the virtual memory subsystem. It seems to me, the VMM would be highly optimized on most systems as it's critical for system performance.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
这实际上取决于您想要做什么。如果您需要做的只是跳到一个已知的偏移量并读出一个小标记，
read()可能会更快（mmap()必须执行一些相当复杂的内部操作会计）。但是，如果您计划复制出 MP3 的全部 200mb，或者扫描它以查找可能出现在未知偏移处的某些标记，那么mmap()可能是一种更快的方法。例如，如果您需要将整个文件向下移动几百个字节以插入 ID3 标记，一种简单的方法是使用 ftruncate() 扩展文件，mmap 文件，然后
memmove()将内容向下一点。但是，如果您的程序在运行时崩溃，这将破坏该文件。您还可以将文件的内容复制到新文件中 - 这是 mmap() 真正发挥作用的另一个地方；您可以简单地mmap()旧文件，然后使用单个write()将其所有数据复制到新文件中。简而言之，如果您要执行大量 IO（就传输的总字节数而言），那么
mmap()非常有用。这是因为它减少了所需的副本数量，并且可以显着减少读取缓存数据所需的内核条目数量。然而，mmap()需要至少两次进入内核（如果完成后清理映射则为三次！），并进行一些复杂的内部内核计算，因此固定开销可以是高的。另一方面，read() 涉及额外的内存到内存复制，因此对于大型 I/O 操作效率较低，但很简单，因此固定开销相对较低。简而言之，对于大批量 I/O 使用
mmap()，对于一次性小 I/O 使用read()或pread()/Os。It really depends on what you're trying to do. If all you need to do is hop to a known offset and read out a small tag,
read()may be faster (mmap()has to do some rather complex internal accounting). If you are planning on copying out all 200mb of the MP3, however, or scanning it for some tag that may appear at an unknown offset, thenmmap()is likely a faster approach.For example, if you need to shift the entire file down a few hundred bytes in order to insert an ID3 tag, one simple approach would be to expand the file with
ftruncate(), mmap the file, thenmemmove()the contents down a bit. This, however, will destroy the file if your program crashes while it's running. You could also copy the contents of the file into a new file - this is another place where mmap() really shines; you can simplymmap()the old file, then copy all of its data into the new file with a singlewrite().In short,
mmap()is great if you're doing a large amount of IO in terms of total bytes transferred; this is because it reduces the number of copies needed, and can significantly reduce the number of kernel entries needed for reading cached data. Howevermmap()requires a minimum of two trips into the kernel (three if you clean up the mapping when you're done!) and does some complex internal kernel accounting, and so the fixed overhead can be high.read()on the other hand involves an extra memory-to-memory copy, and can thus be inefficient for large I/O operations, but is simple, and so the fixed overhead is relatively low. In short, usemmap()for large bulk I/O, andread()orpread()for one-off, small I/Os.不要为
mmap烦恼，除非您的代码受 CPU 限制，特别是由于大量小型读取和写入。mmap可能听起来不错，但这并不是为什么不是每个人都使用这个看起来像的替代方案的原因。鉴于您正在递归访问可能很大的目录结构，您的瓶颈将是目录 IO 和并发性。
mmap不会有帮助。Update0
阅读链接到的问题发现这个答案支持我的经验：
Don't bother with
mmapunless your code is CPU bound, specifically due to lots small reads and writes.mmapmay sound nice, but it isn't the awesome why isn't everyone using this alternative it looks like.Given that you're recursing through potentially large directory structures, your bottleneck will be directory IO and concurrency.
mmapis not going to help.Update0
Reading the linked to question finds this answer that supports my experience:
如果您通常不打算流式传输文件然后处理它，而是跳来跳去（例如读取前面的标签然后跳到末尾等），那么我会使用 mmap ，因为您的代码将更干净、更容易维护，将文件视为一个大缓冲区，而不必自己实际管理缓冲和分页。
正如前面提到的，如果您正在处理大量数据，磁盘 I/O 很可能会主导您的处理。 mmap 可能比 read 快，但对于合理的实现来说，它可能不会快很多，特别是在当今的硬件上，它的速度不断变得越来越快，而磁盘驱动器多年来一直停留在 7200 和 10,000 RPM。
因此，使用 mmap 可以使您的代码变得简单而整洁。
If you're not normally going to be streaming the file in and then processing it, but rather hopping around (like read the tags at the front and then jump to the end, etc.) then I would use mmap simply because your code will be cleaner and easier to maintain treating the file as a large buffer without having to actually manage the the buffering and paging yourself.
As has been mentioned, if you're processing a lot of data disk I/O is likely going to dominate your processing anyway. mmap may be faster than read, but for reasonable implementations, it's likely not THAT much faster, especially on todays hardware which has continually got faster and faster while disk drives have been stuck at 7200 and 10,000 RPM for years and years.
So, go with mmap and make your code easy and neat.
我不知道标准 POSIX 函数是否驻留在您允许的范围内，或者您将用于开发，但请考虑这两个函数：
在
unistd.h中定义，它们可用于截断文件达到指定长度。通过这种方式，您可以轻松我不确定性能，您应该测试此方法，但它应该在内存中加载更少的东西，同时提供一种有意义的方法。
I don't know if standard POSIX functions reside inside what you are allowed or you will to use for the development but think about these two functions:
defined in
unistd.h, which can be used to truncate a file up to a specified length. In this way you could easilyI'm not sure about the performance, you should test this method, but it should load much less things inside ram while providing a senseful way of doing it.