使用 fcntl 锁进行共享 mmap 协调？
当使用 mmap() 共享内存（来自 Linux 或其他类 UNIX 系统）时，是否可以（并且可移植）使用 fcntl() （或 flock() 或 lockf() 函数）来协调对映射的访问？
我的想法是使用进程/页面映射构建共享内存，以最大限度地减少锁定争用。每个进程都可以同时处理其页面，并且仅在更新进程/页面映射时才需要获取锁。 （从无主页面读取访问将涉及检查序列号、复制所需数据，然后验证该块的序列号未更改）。
从概念上讲，共享此文件映射的每个进程都将执行 mmap()，在其中找到一个空闲块，获取进程/页面区域的锁，用自己的分配更新它，释放锁，然后继续愉快地工作。任何进程都可以搜索过时的映射（使用 kill() 以零作为信号）并清理进程/页表映射。
（粗略地说，我正在尝试使用来自 Python 的共享内存而不是 Linux 的生产者/消费者处理引擎；我希望该解决方案能够移植到 BSD 和其他编程语言——只要支持mmap() 以及 fcntl()、flock() 或 lockf() 的必要接口。 I'我也对显示如何测量锁争用并检测任何同步失败的伪代码感兴趣，我知道线程和多处理及其各自的队列。 () 对象是实现 Python 生产者/消费者处理模型的最直接的方法）。
When using mmap() for shared memory (from Linux, or other UNIX-like systems) is it possible (and portable) to use fcntl() (or flock() or lockf() functions) to co-ordinate access to the mapping?
Responses to this SO question seems to suggest that it should work.
The idea I have in mind would be to have the shared memory structured with a process/page map to minimize the locking contention. Processes could each work with their pages concurrently, and a lock would only need to be acquired when updating the process/page mappings. (Read access from unowned pages would involve checking a serial number, copying the desired data, then validating that the serial number of that block hadn't changed).
Conceptually each process sharing this file mapping would perform the mmap(), find a free block therein, acquire a lock to the process/page area, update that with its own assignment, release the lock and then go on merrily with its work. Any process could search for stale mappings (using kill() with zero as the signal) and clean up the process/page table mapping.
(In rough, generic terms, I'm toying with a producer/consumer processing engine using shared memory from Python over Linux; I'd like the solution to be portable to BSD and to other programming languages --- so long as the support mmap() and the necessary interfaces to fcntl(), flock() or lockf(). I'd also be interested in psuedo-code showing how one would measure lock contention and detect any synchronization failures. I am aware that the threading and multiprocessing with their respective Queue() objects are the most straightforward way to implement a Python producer/consumer processing model).
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
我确信锁会提供互斥，但我不知道它们是否会给你带来内存屏障。看起来跳进内核（fcntl、flock 和 lockf 会做的事情）可能会做一些强制无序内存读取和写入提交的事情，但我怀疑你会得到一个硬保证。我认为这是它可能有效的事情之一，测试将表明它确实有效，但你不会知道它总是有效，除非你找到一个参考资料说同样多。
我在 C 中做了类似的事情，但我在共享内存本身中使用了原子自旋锁。过去您必须执行一些内联汇编，但 gcc 现在有一些您可以使用的内部操作：
http://gcc.gnu.org/onlinedocs/gcc/Atomic-Builtins.html
如果您愿意编写一个非常简单的 Python 扩展，您可以包装 __sync_lock_test_and_set( ...) 和 __sync_lock_release(...) 来完成您需要的操作。这些应该非常便携。
我相信也有一种方法可以将 pthread 互斥体放入共享内存中，但我对此没有任何经验。同样，您必须编写一个简单的 C 扩展才能从 Python 访问它。
I'm sure the locks will provide mutual exclusion, but I don't know if they will give you a memory barrier. It seems like jumping into the kernel (which fcntl, flock, and lockf will do) is likely to do something that forces out of order memory reads and writes to commit, but I doubt you'll get a hard guarantee. I think this is one of those things where it probably works, and testing will show that it does work, but you won't know that it always works unless you find a reference saying as much.
I've done something similar to this from C, but I used atomic spinlocks in the shared memory itself. It used to be that you had to do a little bit of inline assembly, but gcc now has some intrinsic operations that you can use:
http://gcc.gnu.org/onlinedocs/gcc/Atomic-Builtins.html
If you're willing to write a very simple Python extension, you could wrap __sync_lock_test_and_set(...) and __sync_lock_release(...) to do what you need. Those should be pretty portable.
I believe there is a way to put pthread mutexes into shared memory too, but I don't have any experience with that. Again, you'd have to write a simple C extension to get access to that from Python.