Python 嵌入线程——避免死锁？
有没有办法嵌入Python，允许从Python回调到C++，允许Python代码生成线程，并避免死锁？
问题是这样的：
要调用 Python，我需要持有 GIL。 通常，我通过在首次创建解释器时获取主线程状态来实现此目的，然后使用 PyEval_RestoreThread() 获取 GIL 并在调用 Python 之前交换线程状态。
当从 Python 调用时，我可能需要访问一些受主机中单独关键部分保护的受保护资源。 这意味着 Python 将持有 GIL（可能来自我最初调用的其他线程），然后尝试获取我的保护锁。
当调用 Python 时，我可能需要持有相同的锁，因为例如，我可能会迭代某些对象集合。
当调用
问题是，即使我在调用 Python 时持有 GIL，Python 也可能会放弃它，将其交给另一个线程，然后让该线程调用我的主机，期望获取主机锁。 同时，主机可以获取主机锁和GIL锁，并调用Python。 僵局随之而来。
这里的问题是，当我调用它时，Python 将 GIL 放弃给另一个线程。 这就是它所期望的，但是它使得顺序锁定变得不可能——即使我首先获取 GIL，然后获取我自己的锁，然后调用 Python，Python 也会从另一个线程调用我的系统，期望获取我自己的锁（因为它通过释放 GIL 对 GIL 进行了取消排序）。
我无法真正让系统的其余部分对系统中所有可能的锁使用 GIL —— 这甚至无法正常工作，因为 Python 仍然可能将其释放给另一个线程。
我也不能真正保证我的主机在进入 Python 时不会持有任何锁，因为我无法控制主机中的所有代码。
那么，是不是只有这样就不能做呢？
Is there any way to embed python, allow callbacks from python to C++, allowing the Pythhon code to spawn threads, and avoiding deadlocks?
The problem is this:
To call into Python, I need to hold the GIL. Typically, I do this by getting the main thread state when I first create the interpreter, and then using PyEval_RestoreThread() to take the GIL and swap in the thread state before I call into Python.
When called from Python, I may need to access some protected resources that are protected by a separate critical section in my host. This means that Python will hold the GIL (potentially from some other thread than I initially called into), and then attempt to acquire my protection lock.
When calling into Python, I may need to hold the same locks, because I may be iterating over some collection of objects, for example.
The problem is that even if I hold the GIL when I call into Python, Python may give it up, give it to another thread, and then have that thread call into my host, expecting to take the host locks. Meanwhile, the host may take the host locks, and the GIL lock, and call into Python. Deadlock ensues.
The problem here is that Python relinquishes the GIL to another thread while I've called into it. That's what it's expected to do, but it makes it impossible to sequence locking -- even if I first take GIL, then take my own lock, then call Python, Python will call into my system from another thread, expecting to take my own lock (because it un-sequenced the GIL by releasing it).
I can't really make the rest of my system use the GIL for all possible locks in the system -- and that wouldn't even work right, because Python may still release it to another thread.
I can't really guarantee that my host doesn't hold any locks when entering Python, either, because I'm not in control of all the code in the host.
So, is it just the case that this can't be done?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
“当调用 Python 时，我可能需要持有相同的锁，因为例如，我可能会迭代某些对象集合。”
这通常表明具有多个线程的单个进程是不合适的。 也许在这种情况下，多个进程（每个进程都有集合中的一个特定对象）更有意义。
独立的进程——每个进程都有自己的线程池——可能更容易管理。
"When calling into Python, I may need to hold the same locks, because I may be iterating over some collection of objects, for example."
This often indicates that a single process with multiple threads isn't appropriate. Perhaps this is a situation where multiple processes -- each with a specific object from the collection -- makes more sense.
Independent process -- each with their own pool of threads -- may be easier to manage.
python 调用的代码应该在获取任何锁之前释放 GIL。
这样我相信就不会陷入僵局。
The code that is called by python should release the GIL before taking any of your locks.
That way I believe it can't get into the dead-lock.
最近 pyopenssl 列表上有一些类似问题的讨论。 恐怕如果我尝试解释这一点，我会弄错，所以我会建议您参考 有问题的问题。
There was recently some discussion of a similar issue on the pyopenssl list. I'm afraid if I try to explain this I'm going to get it wrong, so instead I'll refer you to the problem in question.