这是与子进程共享只读内存的安全方法吗?

发布于 2024-08-20 11:11:19 字数 905 浏览 3 评论 0 原文

我想分配并初始化相当大的连续内存块(~1GB),然后将其标记为只读并分叉多个(比如说几十个)将使用它的子进程,而不制作自己的内存副本(机器没有足够的内存来执行此操作)。

我是否正确地认为,如果我 malloc 内存为通常,然后使用 mprotect(addr, size, PROT_READ) 然后 fork,这将允许子进程安全地使用内存而不导致内存被复制? (前提是我确保在 mprotect 调用之后没有任何内容尝试写入分配的内存)。

编辑:感谢您的所有回答。

后续问题 - 我计划使用 shmget,但是我认为它使用 mm 因此将仅限于较小的分配(请参阅本页的限制部分)。例如 /proc/sys/kernel/shmmax 在我使用的服务器上是 32MB。但我想要 1GB 的连续内存。我对这个限制有错吗?

I want to allocate and initialise a fairly large chunk of contiguous memory (~1GB), then mark it as read-only and fork multiple (say several dozen) child processes which will use it, without making their own copies of the memory (the machine won't have enough memory for this).

Am I right in thinking that if I malloc the memory as usual, then mark it as read-only with mprotect(addr, size, PROT_READ) and then fork, this will allow the child processes to safely use the memory without causing it to be copied? (Providing I ensure nothing tries to write to the allocated memory after the mprotect call).

edit: Thanks for all the answers.

A followup question - I was planning on using shmget, but I thought it used mm and thus would be limited to smaller allocations (see the Restrictions section of this page). eg /proc/sys/kernel/shmmax is 32MB on the server I'm using this one. But I want 1GB of contiguous memory. Am I wrong about this limitation?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(6

爱的故事 2024-08-27 11:11:19

man mprotect

该实现将要求 addrsysconf()

如果映射不是通过调用 mmap()

  1. mprotect 仅适用于页面,而不适用于任意字节范围,因此一般来说 malloc 是不合适的。 posix_memalign 可能有帮助,但是......
  2. 虽然它目前可能在您的系统上运行,但您不应该 mprotect 任何您自己没有 mmap 的内容。请改用 mmap(0, pages*sysconf(_SC_PAGESIZE), PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0)

man mprotect

The implementation will require that addr be a multiple of the page size as returned by sysconf().

The behaviour of this function is unspecified if the mapping was not established by a call to mmap().

  1. mprotect only works on pages, not arbitrary byte ranges, so in general malloc is inappropriate. posix_memalign may help, but...
  2. While it may work on your system at the present time, you shoud not mprotect anything that you didn't mmap yourself. Use mmap(0, pages*sysconf(_SC_PAGESIZE), PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) instead.
贪恋 2024-08-27 11:11:19

您的理由是不正确的,因为任何子进程都可以调用 mprotect() 来删除保护并开始在那里写入。如果页面没有被复制,就会违反fork()的原则。

即使它以写时复制用于分叉进程的方式工作,我也不认为标准中有任何地方这么说(POSIX 没有说它是写时复制,例如)。

您可以使用标准措施来共享内存,而不是使用非标准行为。例如,带有 shm_open 的 POSIX 共享内存以及后续的mmap (正如评论中指出并解释的那样)他的帖子 作者:ephemient)。文件描述符将通过分叉保留。

You are not right by the reason than any of child processes can call mprotect() to remove protection and start writing there. If the pages have not been copied, it would violate the principles of fork().

Even if it works that way that copy-on-write is used for forked processes, I don't reckon any place in standards that says so (POSIX doesn't say it's copy-on-write, for instance).

Instead of using non-standard behavior, you may use standard measures to share memory. For example, the POSIX shared memory with shm_open and consequent mmap (as was pointed out in comment and explained in his post by ephemient). The file descriptor will be preserved through forking.

多孤肩上扛 2024-08-27 11:11:19

无需将其标记为只读,只需让您的子进程不理会它即可。

如果父级和子级都没有写入它,则它应该保持共享。如果你不想改变它,那也没关系。

如果你想写入它,你需要将 mmap 与 MAP_SHARED 一起使用。

There is no need to mark it read-only, just get your child processes to leave it alone.

If neither the parent nor child writes to it, it should remain shared. If you don't ever want to change it, that's fine.

If you want to write to it, you'll want to use mmap with MAP_SHARED.

好倦 2024-08-27 11:11:19

您假设内核将进行写时复制优化,而不复制mprotect-ed页面。但我不会指望它。 malloc-ed 内存周围漂浮着各种元数据 - 保护页等,只有 Ulrich Drepper 知道 libc 内部发生了什么:)

在中准备数据可能会更容易、更安全一个磁盘文件,然后将其 mmap 到所有进程中,或者只是走正常的 POSIX shm_open 路线。

You are making an assumption that the kernel would do copy-on-write optimization and not copy the mprotect-ed pages. I would'n count on it though. malloc-ed memory has all sorts of metadata floating around it - guard pages, etc. etc. and only Ulrich Drepper knows what's going on inside libc :)

It would probably be easier and safer to prepare data in a disk file and them mmap it into all the processes, or just go normal POSIX shm_open route.

め可乐爱微笑 2024-08-27 11:11:19

我的理解是肯定的,因为Linux使用写时复制机制来传递给子进程的内存页面。

My understanding is yes, since Linux uses a copy-on-write mechanism for memory pages passed to a child process.

梦情居士 2024-08-27 11:11:19

你可以这样做。

另一种方法是使用 mmap()

另一种选择是使用 POSIX 共享内存 (shm_open()) ;另一个主要替代方案是 System V 共享内存 (shmget()shmat())。正式共享内存系统的优点之一是您的父进程可以创建内存,然后不相关的进程可以连接到它 - 如果这是有益的。

You could do it that way.

An alternative is to use mmap().

Another alternative is to use the POSIX shared memory (shm_open()); the other main alternative is System V shared memory (shmget(), shmat()). One advantage of the formal shared memory systems is that your parent process can create the memory and then unrelated process could connect to it - if that was beneficial.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文