Python 中的命名信号量?
我有一个 python 脚本,它使用的资源不能被超过一定数量的并发脚本运行所使用。
传统上,这可以通过命名信号量来解决,但我在 multiprocessing< 的文档中找不到这些信号量/a> 模块或 线程 。
我是否遗漏了某些东西或者未由 Python 实现/公开的命名信号量?更重要的是,如果答案是否定的,那么模仿的最佳方法是什么?
谢谢, 波阿斯
PS.由于与这个问题不太相关的原因,我无法将任务聚合到连续运行的进程/守护进程或使用生成的进程 - 这两者似乎都可以与 python API 一起使用。
I have a script in python which uses a resource which can not be used by more than a certain amount of concurrent scripts running.
Classically, this would be solved by a named semaphores but I can not find those in the documentation of the multiprocessing module or threading .
Am I missing something or are named semaphores not implemented / exposed by Python? and more importantly, if the answer is no, what is the best way to emulate one?
Thanks,
Boaz
PS. For reasons which are not so relevant to this question, I can not aggregate the task to a continuously running process/daemon or work with spawned processes - both of which, it seems, would have worked with the python API.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
我建议使用第三方扩展,例如这些,最好是
posix_ipc
- - 特别请参阅文档中的 信号量 部分。这些模块主要是关于以 unixy 方式公开“system V IPC”(包括信号量),但至少其中一个(特别是 posix_ipc)据称可以在 Windows 上与 Cygwin 一起使用(我还没有t 验证了这一说法)。 FreeBSD 7.2 和 Mac OSX 10.5 上有一些记录在案的限制,因此请注意这些平台对你很重要。
I suggest a third party extension like these, ideally the
posix_ipc
one -- see in particular the semaphore section in the docs.These modules are mostly about exposing the "system V IPC" (including semaphores) in a unixy way, but at least one of them (
posix_ipc
specifically) is claimed to work with Cygwin on Windows (I haven't verified that claim). There are some documented limitations on FreeBSD 7.2 and Mac OSX 10.5, so take care if those platforms are important to you.您可以使用文件系统而不是内核路径来模拟它们(无论如何,命名信号量在某些平台上都是以这种方式实现的)。您必须自己实现 sem_[open|wait|post|unlink] ,但这样做应该相对简单。您的同步开销可能会很大(取决于您在应用程序中摆弄信号量的频率),因此您可能需要在启动进程时初始化一个虚拟磁盘来存储命名信号量。
或者,如果您不习惯自己动手,您可以包装
boost::interprocess::named_semaphore
(文档此处)在一个简单的扩展模块中。You can emulate them by using the filesystem instead of a kernel path (named semaphores are implemented this way on some platforms anyhow). You'll have to implement
sem_[open|wait|post|unlink]
yourself, but it ought to be relatively trivial to do so. Your synchronization overhead might be significant (depending on how often you have to fiddle with the semaphore in your app), so you might want to initialize a ramdisk when you launch your process in which to store named semaphores.Alternatively if you're not comfortable rolling your own, you could probably wrap
boost::interprocess::named_semaphore
(docs here) in a simple extension module.