从C++打电话给python lambda时处理吉尔。功能

发布于 2025-02-13 03:04:23 字数 1850 浏览 2 评论 0 原文

问题

是pybind11以某种方式神奇地从事 pygilstate_ensure() and pygilstate_release() ?如果没有,我该怎么做?

更多详细信息

are 许多问题使用PYBIND11函数C ++作为回调,但我还没有找到一个解释GIL与Pybind11的使用。

文档吉尔:

[...]但是,当从C创建线程时(例如,由具有自己的线程管理的第三方库)时,它们不持有GIL,也没有为它们的线程状态结构。

如果您需要从这些线程调用Python代码(通常这将是上述第三方库提供的回调API的一部分),则必须首先通过创建线程状态数据结构来向解释器注册这些线程然后获取GIL,最后存储其线程状态指针,然后才能开始使用Python/C API。

我可以轻松地绑定带有回调的C ++函数:

py::class_<SomeApi> some_api(m, "SomeApi"); 
some_api
    .def(py::init<>())
    .def("mode", &SomeApi::subscribe_mode, "Subscribe to 'mode' updates.");

相应的C ++函数类似:

void subscribe_mode(const std::function<void(Mode mode)>& mode_callback);

但是因为Pybind11无法知道我的C ++实现中发生的线程,所以我想它不能为我处理GIL。因此,如果 mode_callback 是由C ++创建的线程调用的,这是否意味着我应该将包装器写入 someapi :: sisscribe_mode 使用 pygilstate_ensure() /code>和 pygilstate_release()对于每个调用?

<启动/停止线程时,似乎“释放GIL”。我仍然想知道是否存在 py :: call_guard&lt; py :: gil_scoped_acquire&gt;()完全可以按照我(相信我需要的东西,即用 pygilstate_ensure包裹我的回调) () pygilstate_release()

The question

Is pybind11 somehow magically doing the work of PyGILState_Ensure() and PyGILState_Release()? And if not, how should I do it?

More details

There are many questions regarding passing a python function to C++ as a callback using pybind11, but I haven't found one that explains the use of the GIL with pybind11.

The documentation is pretty clear about the GIL:

[...] However, when threads are created from C (for example by a third-party library with its own thread management), they don’t hold the GIL, nor is there a thread state structure for them.

If you need to call Python code from these threads (often this will be part of a callback API provided by the aforementioned third-party library), you must first register these threads with the interpreter by creating a thread state data structure, then acquiring the GIL, and finally storing their thread state pointer, before you can start using the Python/C API.

I can easily bind a C++ function that takes a callback:

py::class_<SomeApi> some_api(m, "SomeApi"); 
some_api
    .def(py::init<>())
    .def("mode", &SomeApi::subscribe_mode, "Subscribe to 'mode' updates.");

With the corresponding C++ function being something like:

void subscribe_mode(const std::function<void(Mode mode)>& mode_callback);

But because pybind11 cannot know about the threading happening in my C++ implementation, I suppose it cannot handle the GIL for me. Therefore, if mode_callback is called by a thread created from C++, does that mean that I should write a wrapper to SomeApi::subscribe_mode that uses PyGILState_Ensure() and PyGILState_Release() for each call?

This answer seems to be doing something similar, but still slightly different: instead of "taking the GIL" when calling the callback, it seems like it "releases the GIL" when starting/stopping the thread. Still I'm wondering if there exists something like py::call_guard<py::gil_scoped_acquire>() that would do exactly what I (believe I) need, i.e. wrapping my callback with PyGILState_Ensure() and PyGILState_Release().

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

噩梦成真你也成魔 2025-02-20 03:04:23

通常,

PYBIND11试图做正确的事情,并且当Pybind11知道它调用Python函数或以C ++代码为单位时,将持有GIL。使用PYBIND11时,您唯一需要明确获取GIL的时间是编写访问Python的C ++代码时,并将从其他C ++代码中调用,或者您已经明确删除了GIL。

std ::功能包装

std :: function 始终通过 gil_scoped_acquire

如果 GIL_SCOPED_ACQUIRE 是从当前没有GIL线程状态关联的线程调用的,则将 gil_scoped_acquire 的destructor释放,然后

如果您只从另一个线程调用一次函数,这不是问题。如果您经常调用回调,它将经常创建/删除线程状态,这可能并不适合性能。最好在线程启动时创建线程状态(甚至更容易,从Python启动线程并从Python调用您的C ++代码)。

In general

pybind11 tries to do the Right Thing and the GIL will be held when pybind11 knows that it is calling a python function, or in C++ code that is called from python via pybind11. The only time that you need to explicitly acquire the GIL when using pybind11 is when you are writing C++ code that accesses python and will be called from other C++ code, or if you have explicitly dropped the GIL.

std::function wrapper

The wrapper for std::function always acquires the GIL via gil_scoped_acquire when the function is called, so your python callback will always be called with the GIL held, regardless which thread it is called from.

If gil_scoped_acquire is called from a thread that does not currently have a GIL thread state associated with it, then it will create a new thread state. As a side effect, if nothing else in the thread acquires the thread state and increments the reference count, then once your function exits the GIL will be released by the destructor of gil_scoped_acquire and then it will delete the thread state associated with that thread.

If you're only calling the function once from another thread, this isn't a problem. If you're calling the callback often, it will create/delete the thread state a lot, which probably isn't great for performance. It would be better to cause the thread state to be created when your thread starts (or even easier, start the thread from Python and call your C++ code from python).

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文