我怎样才能“醒来”?一个事件循环来通知它另一个线程的 Future 已完成?
使用Python async/asyncio时,我经常创建并完成asyncio.future
来自不是线程的线程的对象,这些线程不是线程运行事件循环的线程。
除非我完成正在运行事件循环的线程中的那些未来或通过通知完成循环的函数,否则事件循环通常不会“注意”期货已完成。
是否有一种方法可以“通知”事件循环,如果在外部(通过set_result)外部(通过set_result)在外部(通过SET_RESULT)检查未来是否应该检查未来?
为什么我要问这个
线程)延迟非常低,b)检查是否已同步,以后同步(例如future.done.done()
)。
事件循环等待期货不是 需要较低的延迟,以通知他们准备就绪 - 可以迟到几毫秒通知几毫秒。
理想情况下,将有一种表现方式来通知事件循环,即在线程中同步准备后就准备好了未来。
即使不可能,只要期货在线程中尽快完成期货,事件循环也可以在间隔内进行轮询准备。
我尝试了
解决此问题的“正确”方法是call_soon_threadsafe
,例如:
def do_in_thread(future):
future.get_loop().call_soon_threasafe(future.set_result, "the result")
它通知事件循环未来的准备就绪,但没有
- 有两个原因(8--) 。 10x)开销与调用
future.set_result
在我的基准测试中。 - 在事件循环运行之前,它没有准备好未来,这意味着我无法可靠地检查未来是否完成,我需要做。例如,这是行不通的:
def do_in_thread(future):
future.get_loop().call_soon_threasafe(future.set_result, "the result")
assert future.done() # Fails
看起来 工作的一件事是通过故意失败a second 调用set_result 通过
call_soon_threadsafe
,然后吞咽invalidStateError
,像这样:
def ensure_result(f, res):
try:
f.set_result(res)
except InvalidStateError:
pass
def in_thread(fut: Future):
fut.set_result("the result")
fut.get_loop().call_soon_threadsafe(ensure_result, fut, "the result")
仍然有开销,但是我可以通过跟踪未来的未来来删除调用call_soon_threadsafe
在线程共享的数据结构和轮询suars_result
中,偶尔会呼叫。但是,我仍然不确定:
- 这可靠地起作用吗?是
set_result
invalidStateError
保证通知事件循环的等待
给定的未来可以返回等待
,还是我依靠的无证件实现细节? - 有没有更好的方法来实现这一定期罢工,这并不涉及我自己跟踪/轮询这样的未来?
在一个完美的世界中,会有一个loop.poll_all_pending_futures()
或loop.update_future_state(fut)
方法可以有效地实现这一目标,但我不知道一个。
When using python async/asyncio, I often create and complete asyncio.Future
objects from threads that are not the thread running the event loop.
Unless I complete those futures in the thread that is running the event loop or via a function that notifies that loop of the completion, the event loop often does not "notice" that the futures are completed.
Is there a way to "notify" an event loop that it should check a Future for completion if that future was readied (via set_result) externally?
Why I am asking this
The threads which ready futures need to a) have very low latency, and b) check whether the future has been readied, synchronously, later on (e.g. via future.done()
).
The event loop await
ing the Futures does not need to have low latency in being notified that they're ready--it can be notified a few milliseconds late.
Ideally there would be a performant way to notify the event loop that a Future had been readied after readying it synchronously in a thread.
Even if that's not possible, the event loop could poll readiness on an interval, so long as the futures were synchronously readied as quickly as possible in threads.
What I have tried
The "correct" way to solve this problem is with call_soon_threadsafe
, e.g.:
def do_in_thread(future):
future.get_loop().call_soon_threasafe(future.set_result, "the result")
That notifies the event loop of Future readiness reliably, but does not work for two reasons:
- It has significant (8-10x) overhead versus calling
future.set_result
in my benchmarks. - It doesn't ready the Future until the event loop runs, which means I can't reliably check if the Future is done, which I need to do. For example, this won't work:
def do_in_thread(future):
future.get_loop().call_soon_threasafe(future.set_result, "the result")
assert future.done() # Fails
One thing that does seem to work is to notify the event loop by intentionally failing a second call to set_result
via call_soon_threadsafe
, and swallowing the InvalidStateError
, like this:
def ensure_result(f, res):
try:
f.set_result(res)
except InvalidStateError:
pass
def in_thread(fut: Future):
fut.set_result("the result")
fut.get_loop().call_soon_threadsafe(ensure_result, fut, "the result")
That still has overhead, but I could remove the overhead of calling call_soon_threadsafe
by tracking Futures in a thread-shared data structure and polling calls to ensure_result
occasionally. However, I'm still not sure:
- Does that reliably work? Is
set_result
failing withInvalidStateError
guaranteed to notify the event loop that aawait
given Future can return out ofawait
, or is that an undocumented implementation detail I'm relying on? - Is there a better way to achieve that periodic-wakeup that doesn't involve me keeping track of/polling such Futures myself?
In a perfect world, there would be a loop.poll_all_pending_futures()
or loop.update_future_state(fut)
method which would achieve this efficiently, but I don't know of one.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
我遇到了同样的基本问题,并且几天来我的头撞在墙上。我使用 C++ 扩展中的回调通过在另一个系统线程(在 C 端创建)中运行的代码来设置未来结果。我可以设置未来的结果,但事件循环并不关心,除非我有其他协程使其在该 Python 线程中保持“活动”,即使如此,它通常也很慢。
我使用这个概念解决方案(针对您的示例代码发布)来解决它:
结果是在设置未来和让事件循环“唤醒”并处理它之间有大约 100 微秒(不是毫,微)的延迟。
I ran into the same essential problem, and banged my head on the wall for a few days. I was using callbacks from a C++ extension to set the future result via code running in another system thread (created on the C side). I could set that future result, but the event loop didn't care unless I had other coroutines keeping it "alive" in that Python thread, and even then it was often slow.
I used this conceptual solution (posting against your example code) to solve it:
The result was a delay of like 100 microseconds (not milli, micro) between setting the future and getting the event loop to "wake up" and process it.