Use shm_open to create a named shared memory object, followed by ftruncate to set the size you need. You can then mmap part or all of it for writing, close it, and again shm_open it (using the same name) and mmap it in another process later. Once you're done with it, you can shm_unlink it.
根据您提到的 NFS 进行更新。在 Linux 中查找缓存设置,将其设置得非常非常积极,以便内核尽可能进行缓存并避免写回磁盘(或 NFS)。
Use a regular file, but memory map it. That way, the second process can access it just as easily as the first. The OS caching will take care of keeping the "hot" parts of the file in RAM.
Update, based on your mention of NFS. Look for caching settings in Linux, increase them to be very, very aggressive, so the kernel caches and avoids writing back to disk (or NFS) as much as possible.
The solution for you would be to use use shared memory (e.g. with mmap). To circumvent the problem that your two process do not run at the same time introduce an additional process (call it the "ramdisk"-process). That runs permanent and keeps the memory map alive, while your other process can connect to it.
Usually you setup a ram-disk using admin tools and use it in your program as a normal filesystem. To share data between different processes you could use shared-memory. I'm not sure what you want to achieve by loading 150GB into memory (are you sure you have that much RAM?).
Ten years ago, I tried to put c-header files into a ram-disk to speed-up compilation, unfortunatly this had no measureable effect, because the normal file system caches them already.
发布评论
评论(4)
使用
shm_open
创建一个named共享内存对象,然后使用ftruncate
设置您需要的大小。然后,您可以mmap
部分或全部内容进行写入,关闭
它,然后再次shm_open
它(使用相同的名称)和稍后将其映射到另一个进程中。完成后,您可以
shm_unlink
它。Use
shm_open
to create a named shared memory object, followed byftruncate
to set the size you need. You can thenmmap
part or all of it for writing,close
it, and againshm_open
it (using the same name) andmmap
it in another process later. Once you're done with it, you canshm_unlink
it.使用常规文件,但内存映射它。这样,第二个进程就可以像第一个进程一样轻松地访问它。操作系统缓存将负责将文件的“热”部分保存在 RAM 中。
根据您提到的 NFS 进行更新。在 Linux 中查找缓存设置,将其设置得非常非常积极,以便内核尽可能进行缓存并避免写回磁盘(或 NFS)。
Use a regular file, but memory map it. That way, the second process can access it just as easily as the first. The OS caching will take care of keeping the "hot" parts of the file in RAM.
Update, based on your mention of NFS. Look for caching settings in Linux, increase them to be very, very aggressive, so the kernel caches and avoids writing back to disk (or NFS) as much as possible.
您的解决方案是使用共享内存(例如使用 mmap)。为了避免两个进程不同时运行的问题,请引入一个附加进程(称为“ramdisk”进程)。它会永久运行并保持内存映射处于活动状态,而您的其他进程可以连接到它。
The solution for you would be to use use shared memory (e.g. with mmap). To circumvent the problem that your two process do not run at the same time introduce an additional process (call it the "ramdisk"-process). That runs permanent and keeps the memory map alive, while your other process can connect to it.
通常,您使用管理工具设置 RAM 磁盘,并将其作为普通文件系统在程序中使用。要在不同进程之间共享数据,可以使用共享内存。
我不确定你想通过将 150GB 加载到内存中来实现什么(你确定你有那么多 RAM 吗?)。
十年前,我尝试将c头文件放入内存盘中以加速编译,不幸的是这没有可测量的效果,因为普通文件系统已经缓存了它们。
Usually you setup a ram-disk using admin tools and use it in your program as a normal filesystem. To share data between different processes you could use shared-memory.
I'm not sure what you want to achieve by loading 150GB into memory (are you sure you have that much RAM?).
Ten years ago, I tried to put c-header files into a ram-disk to speed-up compilation, unfortunatly this had no measureable effect, because the normal file system caches them already.