fork() 写时复制是一种稳定的公开行为,可用于实现只读共享内存吗?
fork() 的手册页指出它不复制数据页,而是将它们映射到子进程并放置一个写时复制标志。这种行为
- 在 Linux 风格之间是否一致?
- 考虑了实施细节并因此可能会改变?
我想知道是否可以使用 fork() 作为廉价获取共享只读内存块的方法。如果内存是物理复制的,那么它会相当昂贵 - 正在进行很多分叉,并且数据区域足够大 - 但我希望不会......
The man page on fork() states that it does not copy data pages, it maps them into the child process and puts a copy-on-write flag. Is that behavior:
- consistent between flavors of Linux?
- considered an implementation detail and therefore likely to change?
I'm wondering if I can use fork() as a means to get a shared read-only memory block on the cheap. If the memory is physically copied, it would be rather expensive - there's a lot of forking going on, and the data area is big enough - but I'm hoping not...
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(5)
是的
所有 Linux 发行版都使用相同的内核,尽管其版本和发行版略有不同。
另一个底层 fork(2) 实现不太可能很快就会变得更快,因此可以肯定的是,写时复制将继续成为这种机制。也许它不会永远,但肯定会持续很多年。
当然,一些主要的软件系统(例如,Phusion Passenger)在同一系统中使用 fork(2)以您想要的方式进行,这样您就不会是唯一利用 CoW 的人。
Yes
All the linux distros use the same kernel, albeit with slightly different versions and releases of it.
It's unlikely that another underlying fork(2) implementation will be faster any time soon, so it's a safe bet that copy-on-write will continue to be the mechanism. Perhaps it won't be forever, but for years, definitely.
Certainly some major software systems (for example, Phusion Passenger) use fork(2) in the same way that you want to, so you would not be the only one taking advantage of CoW.
这种方法的成功取决于您对自我施加的“只读”限制的坚持程度。父母和孩子都必须遵守这一限制,否则记忆就会被复制。
然而,这可能不是您想象的灾难。内核可以复制小至单个页面(通常为 4 KB)来实现 CoW 语义。典型的 Linux 服务器将使用更复杂的东西,某种 slab 分配器,因此复制的区域可能会大得多。
要点是,这与程序的内存使用概念脱钩。如果您
malloc()
1 GB RAM,分叉出一个子内存块,并且该子内存块仅更改该内存块的第一个字节,则不会复制整个 1 GB 块。也许只复制一页,直到包含第一个字节的slab大小。The success of this approach depends on how well you stick to your self-imposed "read-only" limitation. Both parent and child have to obey this stricture, else the memory gets copied.
This may not be the catastrophe you're envisioning, however. The kernel can copy as little as a single page (typically 4 KB) to implement CoW semantics. A typical Linux server will use something more complex, some sort of slab allocator, so the copied region could be much larger.
The main point is that this is decoupled from your program's conception of its memory use. If you
malloc()
1 GB of RAM, fork off a child, and the child changes just the first byte of that memory block, the entire 1 GB block isn't copied. Perhaps as little as one page is copied, up to the slab size containing that first byte.您能相信所有 Linux 版本都是这样做的吗?不会。但您可以相信,那些不使用更快方法的人。
因此,您应该使用该功能并依赖它,并在遇到性能问题时重新考虑您的决定。
Can you rely on the fact that all Linux flavors do it this way? No. But you can rely on the fact that those who don't use an even faster method.
Therefore you should use the feature and rely on it and revisit your decision if you get a performance problem.
是的,您当然可以依赖 MMU-Linux 内核;这几乎就是一切。
但是,页面大小并非到处都相同。
通过使用 mmap() 创建匿名映射(不受物理文件支持的映射),可以显式为分叉进程创建共享内存区域。在分叉上,该区域将始终保持共享(前提是子进程没有取消映射它,或者在同一地址映射其他内容)。如果需要,您可以将其保护为只读。
使用(例如)malloc分配的内存很容易最终与非只读的东西共享一个页面,这意味着当另一个结构被修改时它无论如何都会被复制。这包括 malloc 实现使用的内部结构。因此,您可能希望为此目的映射特定区域并从中进行分配。
Yes you can certainly rely on it on MMU-Linux kernels; this is almost everything.
However, the page size isn't the same everywhere.
It is possible to explicitly make a shared memory area for forked process, by using mmap() to create an anonymous map - one which is not backed by a physical file. On fork, this area will always remain shared (provided the child doesn't unmap it, or map something else in at the same address). You can mprotect it to be readonly if you want.
Memory allocated with (for example) malloc can easily end up sharing a page with something that isn't readonly, which means it gets copied anyway when another structure is modified. This includes internal structures used by the malloc implementation. So you might want to mmap a specific area for this purpose and allocate from that.
在没有 MMU(内存管理单元)的机器上运行的 Linux 将在 fork() 上复制所有进程内存。
然而,这些系统通常非常小并且是嵌入式的,您可能不必担心它们。
许多服务(例如 Apache 的 fork 模型)使用initialize 和 fork() 方法来共享初始化的数据结构。
您应该意识到,如果您使用 Perl 和 Python 等使用引用计数变量的语言,或 C++ shared_ptr 的语言,则此模型将不起作用。它不起作用,因为随着引用计数的上下调整,内存变得不共享并被复制。
这会导致 Perl 守护进程(例如尝试使用初始化和分叉模型的 SpamAssassin)使用大量内存。
Linux running on machines without a MMU (memory management unit) will copy all process memory on fork().
However, those systems are usually very small and embedded and you probably don't have to worry about them.
Many services such as Apache's fork model, use the initialize and fork() method to share initialized data structures.
You should be aware that if you are using languages like Perl and Python that use reference-counted variables, or C++ shared_ptr's, this model will not work. It will not work because as the reference counts are adjusted up and down, the memory becomes unshared and gets copied.
This causes huge amounts of memory usage in Perl daemons like SpamAssassin that attempt to use an initialize and fork model.