如何从线程中获取返回值?
下面的函数 foo
返回一个字符串 'foo'
。如何获取从线程目标返回的值 'foo'
?
from threading import Thread
def foo(bar):
print('hello {}'.format(bar))
return 'foo'
thread = Thread(target=foo, args=('world!',))
thread.start()
return_value = thread.join()
上面所示的“一种明显的方法”不起作用:thread.join()
返回 None
。
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(30)
我见过的一种方法是将可变对象(例如列表或字典)以及索引或其他某种标识符传递给线程的构造函数。然后,线程可以将其结果存储在该对象的专用槽中。例如:
如果您确实希望
join()
返回被调用函数的返回值,您可以使用Thread
子类来完成此操作,如下所示:由于一些名称修改,它变得毛茸茸的,并且它访问特定于 Thread 实现的“私有”数据结构......但它有效。
对于Python 3:
One way I've seen is to pass a mutable object, such as a list or a dictionary, to the thread's constructor, along with a an index or other identifier of some sort. The thread can then store its results in its dedicated slot in that object. For example:
If you really want
join()
to return the return value of the called function, you can do this with aThread
subclass like the following:That gets a little hairy because of some name mangling, and it accesses "private" data structures that are specific to
Thread
implementation... but it works.For Python 3:
FWIW,
multiprocessing
模块使用Pool
类为此提供了一个很好的接口。如果您想坚持使用线程而不是进程,则可以使用 multiprocessing.pool.ThreadPool 类作为直接替代品。FWIW, the
multiprocessing
module has a nice interface for this using thePool
class. And if you want to stick with threads rather than processes, you can just use themultiprocessing.pool.ThreadPool
class as a drop-in replacement.在 Python 3.2+ 中,stdlib
concurrent.futures
模块为
线程
提供了更高级别的 API,包括将返回值或异常从工作线程传递回主线程。您可以调用result()
方法 在Future
实例上,它将等到线程完成后再返回线程函数的结果值。In Python 3.2+, stdlib
concurrent.futures
module provides a higher level API tothreading
, including passing return values or exceptions from a worker thread back to the main thread. You can call theresult()
method on aFuture
instance, and it will wait until the thread is completed before returning the result value of the thread's function.Jake的答案很好,但是如果您不想使用线程池(您不知道需要多少个线程,但根据需要创建它们),那么在线程之间传输信息的一个好方法是内置Queue.Queue 类,因为它提供线程安全性。
我创建了以下装饰器,使其以与线程池类似的方式运行:
然后您只需将其用作:
装饰函数每次调用时都会创建一个新线程,并返回一个 Thread 对象,该对象包含将接收结果的队列。
更新
自从我发布这个答案以来已经有一段时间了,但它仍然得到了浏览,所以我想我会更新它以反映我在较新版本的Python中执行此操作的方式:
Python 3.2添加在< href="https://docs.python.org/3/library/concurrent.futures.html" rel="noreferrer">
concurrent.futures
模块提供了一个高级并行电平接口任务。它提供了 ThreadPoolExecutor 和 ProcessPoolExecutor,因此您可以使用具有相同 api 的线程或进程池。此 api 的好处之一是向
Executor
提交任务会返回Future
对象,它将以您提交的可调用对象的返回值完成。这使得附加一个队列对象变得不必要,这大大简化了装饰器:
如果没有传入,这将使用默认的模块线程池执行器。
用法非常相似之前:
如果您使用的是 Python 3.4+,使用此方法(以及一般的 Future 对象)的一个非常好的功能是,返回的 future 可以被包装以将其转换为
asyncio.Future
与asyncio.wrap_future
。这使得它可以轻松地与协程一起工作:如果您不需要访问底层的concurrent.Future对象,您可以将包装包含在装饰器中:
然后,每当您需要推动CPU密集型或阻塞时事件循环线程之外的代码,您可以将其放入装饰函数中:
Jake's answer is good, but if you don't want to use a threadpool (you don't know how many threads you'll need, but create them as needed) then a good way to transmit information between threads is the built-in Queue.Queue class, as it offers thread safety.
I created the following decorator to make it act in a similar fashion to the threadpool:
Then you just use it as:
The decorated function creates a new thread each time it's called and returns a Thread object that contains the queue that will receive the result.
UPDATE
It's been quite a while since I posted this answer, but it still gets views so I thought I would update it to reflect the way I do this in newer versions of Python:
Python 3.2 added in the
concurrent.futures
module which provides a high-level interface for parallel tasks. It providesThreadPoolExecutor
andProcessPoolExecutor
, so you can use a thread or process pool with the same api.One benefit of this api is that submitting a task to an
Executor
returns aFuture
object, which will complete with the return value of the callable you submit.This makes attaching a
queue
object unnecessary, which simplifies the decorator quite a bit:This will use a default module threadpool executor if one is not passed in.
The usage is very similar to before:
If you're using Python 3.4+, one really nice feature of using this method (and Future objects in general) is that the returned future can be wrapped to turn it into an
asyncio.Future
withasyncio.wrap_future
. This makes it work easily with coroutines:If you don't need access to the underlying
concurrent.Future
object, you can include the wrap in the decorator:Then, whenever you need to push cpu intensive or blocking code off the event loop thread, you can put it in a decorated function:
另一种不需要更改现有代码的解决方案:
它也可以轻松调整到多线程环境:
Another solution that doesn't require changing your existing code:
It can be also easily adjusted to a multi-threaded environment:
更新:
我认为有一种更简单、更简洁的方法来保存线程的结果,并且可以使接口与
threading.Thread
类保持几乎相同(如果有边缘情况,请告诉我 - 我没有像下面的原始帖子那样进行测试):为了稳健并避免潜在错误:
简短说明:我们覆盖仅
threading.Thread
的run
方法,并且其他不做修改。这使我们能够使用 threading.Thread 类为我们所做的所有其他事情,而无需担心丢失潜在的边缘情况,例如_private
属性赋值 或 自定义属性修改 按照我原来的帖子的方式。我们可以通过查看
help(ConciseResult)
和help(ConciseRobustResult)
的输出来验证我们是否只修改了run
方法。此处定义的方法: 下包含的唯一方法/属性/描述符是run
,其他所有内容都来自继承的threading.Thread
基类(请参阅从 threading.Thread 继承的方法:
部分)。要使用下面的示例代码测试其中任一实现,请在
main
函数中用ConciseResult
或ConciseRobustResult
替换ThreadWithResult
以下。在
init
方法中使用闭包函数的原始帖子:我发现的大多数答案都很长,需要熟悉其他模块或高级Python功能,并且会让某些人感到相当困惑,除非他们已经熟悉答案所讨论的所有内容。
简化方法的工作代码:
示例代码:
说明:
我想大大简化事情,因此我创建了一个 ThreadWithResult 类并让它继承自 threading.Thread 。
__init__
中的嵌套函数function
调用我们要保存其值的线程函数,并将该嵌套函数的结果保存为实例属性self.result
线程完成执行后。创建它的实例与创建
threading.Thread
的实例相同。将要在新线程上运行的函数传递给target
参数,将函数可能需要的任何参数传递给args
参数,并将任何关键字参数传递给>kwargs
参数。例如,
我认为这比绝大多数答案更容易理解,并且这种方法不需要额外的导入!我包含了
time
和random
模块来模拟线程的行为,但它们不需要实现 原始问题。我知道我在问题提出后很长时间才回答这个问题,但我希望这将来可以帮助更多的人!
编辑:我创建了
save-thread-result PyPI 包
,允许您访问上面相同的代码并在项目之间重用它(GitHub代码在这里)。 PyPI 包完全扩展了
threading.Thread
类,因此您可以在ThreadWithResult
类的threading.thread
上设置任何属性,如下所示出色地!上面的原始答案概述了这个子类背后的主要思想,但有关更多信息,请参阅 更详细的解释(来自模块文档字符串)位于此处。
快速使用示例:
UPDATE:
I think there's a significantly simpler and more concise way to save the result of the thread, and in a way that keeps the interface virtually identical to the
threading.Thread
class (please let me know if there are edge cases - I haven't tested as much as my original post below):To be robust and avoid potential errors:
Short explanation: we override only the
run
method ofthreading.Thread
, and modify nothing else. This allows us to use everything else thethreading.Thread
class does for us, without needing to worry about missing potential edge cases such as_private
attribute assignments or custom attribute modifications in the way that my original post does.We can verify that we only modify the
run
method by looking at the output ofhelp(ConciseResult)
andhelp(ConciseRobustResult)
. The only method/attribute/descriptor included underMethods defined here:
isrun
, and everything else comes from the inheritedthreading.Thread
base class (see theMethods inherited from threading.Thread:
section).To test either of these implementations using the example code below, substitute
ConciseResult
orConciseRobustResult
forThreadWithResult
in themain
function below.Original post using a closure function in the
init
method:Most answers I've found are long and require being familiar with other modules or advanced python features, and will be rather confusing to someone unless they're already familiar with everything the answer talks about.
Working code for a simplified approach:
Example code:
Explanation:
I wanted to simplify things significantly, so I created a
ThreadWithResult
class and had it inherit fromthreading.Thread
. The nested functionfunction
in__init__
calls the threaded function we want to save the value of, and saves the result of that nested function as the instance attributeself.result
after the thread finishes executing.Creating an instance of this is identical to creating an instance of
threading.Thread
. Pass in the function you want to run on a new thread to thetarget
argument and any arguments that your function might need to theargs
argument and any keyword arguments to thekwargs
argument.e.g.
I think this is significantly easier to understand than the vast majority of answers, and this approach requires no extra imports! I included the
time
andrandom
module to simulate the behavior of a thread, but they're not required to achieve the functionality asked in the original question.I know I'm answering this looong after the question was asked, but I hope this can help more people in the future!
EDIT: I created the
save-thread-result
PyPI package to allow you to access the same code above and reuse it across projects (GitHub code is here). The PyPI package fully extends thethreading.Thread
class, so you can set any attributes you would set onthreading.thread
on theThreadWithResult
class as well!The original answer above goes over the main idea behind this subclass, but for more information, see the more detailed explanation (from the module docstring) here.
Quick usage example:
Parris / kindall 的 answer
join
/return
答案已移植到 Python 3 :注意,
Thread
类在 Python 3 中的实现方式有所不同。Parris / kindall's answer
join
/return
answer ported to Python 3:Note, the
Thread
class is implemented differently in Python 3.我偷了kindall的答案并稍微清理了一下。
关键部分是向 join() 添加 *args 和 **kwargs 以处理超时
下面更新的答案
这是我最受欢迎的答案,所以我决定使用将运行的代码进行更新py2 和 py3。
此外,我看到这个问题的许多答案都表明缺乏对 Thread.join() 的理解。有些完全无法处理超时参数。但是,当您有 (1) 一个可以返回
None
的目标函数并且 (2) 您还传递了timeout
时,您还应该注意有关实例的极端情况。代码> arg 到 join()。请参阅“测试 4”以了解这个极端情况。与 py2 和 py3 一起使用的 ThreadWithReturn 类:
下面显示了一些示例测试:
您能确定我们在 TEST 4 中可能遇到的极端情况吗?
问题是我们期望 GiveMe() 返回 None (参见测试 2),但我们也期望 join() 在超时时返回 None。
returned is None
意味着:(1) 这就是 GiveMe() 返回的内容,或者
(2) join() 超时
这个例子很简单,因为我们知道 GiveMe() 将始终返回 None。但在现实世界的实例中(目标可能合法地返回 None 或其他内容),我们希望明确检查发生了什么。
以下是解决这个极端情况的方法:
I stole kindall's answer and cleaned it up just a little bit.
The key part is adding *args and **kwargs to join() in order to handle the timeout
UPDATED ANSWER BELOW
This is my most popularly upvoted answer, so I decided to update with code that will run on both py2 and py3.
Additionally, I see many answers to this question that show a lack of comprehension regarding Thread.join(). Some completely fail to handle the
timeout
arg. But there is also a corner-case that you should be aware of regarding instances when you have (1) a target function that can returnNone
and (2) you also pass thetimeout
arg to join(). Please see "TEST 4" to understand this corner case.ThreadWithReturn class that works with py2 and py3:
Some sample tests are shown below:
Can you identify the corner-case that we may possibly encounter with TEST 4?
The problem is that we expect giveMe() to return None (see TEST 2), but we also expect join() to return None if it times out.
returned is None
means either:(1) that's what giveMe() returned, or
(2) join() timed out
This example is trivial since we know that giveMe() will always return None. But in real-world instance (where the target may legitimately return None or something else) we'd want to explicitly check for what happened.
Below is how to address this corner-case:
使用队列:
Using Queue :
我发现执行此操作的最短、最简单的方法是利用 Python 类及其动态属性。您可以使用
threading.current_thread()
从生成的线程的上下文中检索当前线程,并将返回值分配给属性。The shortest and simplest way I've found to do this is to take advantage of Python classes and their dynamic properties. You can retrieve the current thread from within the context of your spawned thread using
threading.current_thread()
, and assign the return value to a property.我解决这个问题的方法是将函数和线程包装在一个类中。不需要使用池、队列或 c 类型变量传递。它也是非阻塞的。您改为检查状态。请参阅代码末尾如何使用它的示例。
My solution to the problem is to wrap the function and thread in a class. Does not require using pools,queues, or c type variable passing. It is also non blocking. You check status instead. See example of how to use it at end of code.
考虑到 @iman 对 @JakeBiesinger 答案的评论,我已将其重新组合为具有不同数量的线程:
Taking into consideration @iman comment on @JakeBiesinger answer I have recomposed it to have various number of threads:
我正在使用这个包装器,它可以轻松地将任何函数转变为在线程中运行 - 处理其返回值或异常。它不会增加
队列
开销。使用示例
关于
threading
模块的注释舒适的返回值和返回值线程函数的异常处理是一种常见的“Pythonic”需求,并且确实应该由
threading
模块提供 - 可能直接在标准Thread
类中提供。ThreadPool
对于简单任务来说开销太大 - 3 个管理线程,很多官僚机构。不幸的是,Thread
的布局最初是从 Java 复制的 - 例如,您可以从仍然无用的第一个(!)构造函数参数group
中看到这一点。I'm using this wrapper, which comfortably turns any function for running in a
Thread
- taking care of its return value or exception. It doesn't addQueue
overhead.Usage Examples
Notes on
threading
moduleComfortable return value & exception handling of a threaded function is a frequent "Pythonic" need and should indeed already be offered by the
threading
module - possibly directly in the standardThread
class.ThreadPool
has way too much overhead for simple tasks - 3 managing threads, lots of bureaucracy. UnfortunatelyThread
's layout was copied from Java originally - which you see e.g. from the still useless 1st (!) constructor parametergroup
.根据所提到的内容,这是适用于 Python3 的更通用的解决方案。
用法
Based of what kindall mentioned, here's the more generic solution that works with Python3.
Usage
join
总是返回None
,我认为你应该子类Thread
来处理返回码等。join
always returnNone
, i think you should subclassThread
to handle return codes and so.您可以在线程函数的范围之上定义一个可变的,并将结果添加到其中。 (我还修改了代码以兼容 python3)
这将返回
{'world!': 'foo'}
如果您使用函数输入作为结果字典的键,则保证每个唯一输入在结果中输入一个条目
You can define a mutable above the scope of the threaded function, and add the result to that. (I also modified the code to be python3 compatible)
This returns
{'world!': 'foo'}
If you use the function input as the key to your results dict, every unique input is guaranteed to give an entry in the results
您可以使用
ThreadPool()
的pool.apply_async()
来返回值 来自test()
如下所示:并且,您还可以使用
submit()
的concurrent.futures.ThreadPoolExecutor()
从test()
返回值如下所示:,您可以使用数组
result
来代替return
,如下所示:并且
返回
,您还可以使用队列结果
,如下所示:You can use
pool.apply_async()
ofThreadPool()
to return the value fromtest()
as shown below:And, you can also use
submit()
ofconcurrent.futures.ThreadPoolExecutor()
to return the value fromtest()
as shown below:And, instead of
return
, you can use the arrayresult
as shown below:And instead of
return
, you can also use the queueresult
as shown below:这是一个相当老的问题,但我想分享一个对我有用并帮助我的开发过程的简单解决方案。
这个答案背后的方法论是这样一个事实:“新”目标函数
inner
正在将原始函数的结果(通过__init__
函数传递)分配给>result
包装器的实例属性通过称为闭包的方式实现。这允许包装类保留返回值以供调用者随时访问。
注意:此方法不需要使用任何损坏的方法或
threading.Thread
类的私有方法,尽管尚未考虑yield 函数(OP 未提及yield 函数)。享受!
此外,您可以使用以下代码运行 pytest(假设您已安装)来演示结果:
This is a pretty old question, but I wanted to share a simple solution that has worked for me and helped my dev process.
The methodology behind this answer is the fact that the "new" target function,
inner
is assigning the result of the original function (passed through the__init__
function) to theresult
instance attribute of the wrapper through something called closure.This allows the wrapper class to hold onto the return value for callers to access at anytime.
NOTE: This method doesn't need to use any mangled methods or private methods of the
threading.Thread
class, although yield functions have not been considered (OP did not mention yield functions).Enjoy!
Additionally, you can run
pytest
(assuming you have it installed) with the following code to demonstrate the results:将您的目标定义为
1) 接受一个参数
q
2) 将任何语句
return foo
替换为q.put(foo); return
这样一个函数
就会变成
,然后你就可以继续
这样你可以使用函数装饰器/包装器来制作它,这样你就可以使用现有的函数作为
目标
而不修改它们,但遵循这个基本方案。Define your target to
1) take an argument
q
2) replace any statements
return foo
withq.put(foo); return
so a function
would become
and then you would proceed as such
And you can use function decorators/wrappers to make it so you can use your existing functions as
target
without modifying them, but follow this basic scheme.GuySoft的想法很棒,但我认为该对象不一定必须从Thread继承,并且start()可以从接口中删除:
GuySoft's idea is great, but I think the object does not necessarily have to inherit from Thread and start() could be removed from interface:
如前所述,多处理池比基本线程慢得多。按照此处一些答案中的建议使用队列是一种非常有效的替代方案。我将它与字典一起使用,以便能够运行大量小线程并通过将它们与字典组合来恢复多个答案:
As mentioned multiprocessing pool is much slower than basic threading. Using queues as proposeded in some answers here is a very effective alternative. I have use it with dictionaries in order to be able run a lot of small threads and recuperate multiple answers by combining them with dictionaries:
这是我根据@Kindall 的回答创建的版本。
这个版本使得您所要做的就是输入带有参数的命令来创建新线程。
这是用 Python 3.8 制作的:
Here is the version that I created of @Kindall's answer.
This version makes it so that all you have to do is input your command with arguments to create the new thread.
This was made with Python 3.8:
感谢@alec-cureau。下面的代码对我来说工作正常。
替换已弃用的 currentThread():
Thanks to @alec-cureau. below code is working fine for me.
Replaced deprecated currentThread():
一种常见的解决方案是用类似这样的装饰器包装函数
foo
那么整个代码可能看起来像这样
注意
一个重要的问题是返回值可能是unorderred。
(事实上,
返回值
不一定保存到队列
中,因为你可以选择任意线程安全数据结构)One usual solution is to wrap your function
foo
with a decorator likeThen the whole code may looks like that
Note
One important issue is that the return values may be unorderred.
(In fact, the
return value
is not necessarily saved to thequeue
, since you can choose arbitrary thread-safe data structure )Python3 中的 Kindall 的回答
Kindall's answer in Python3
我不知道这是否对你们有用,但我选择创建
全局对象[主要是字典或嵌套数组],这样函数可以访问该对象并对其进行变异,我知道这需要更多资源,但我们不处理量子科学,所以,我想我们可以提供更多的内存假设 RAM 消耗随 CPU 使用率线性增加。
下面是一个示例代码:
该程序运行 4 个线程,每个线程返回一些文本 L[i] for i in L 的数据,从 API 返回的数据存储在字典中,
无论是否有益,每个程序都可能有所不同,对于中小型繁重的计算任务,此对象突变工作得相当快,并且使用了更多资源的几个百分比。
I don't Know whether or not this worked for you guys but I choose to create
a global object [mostly dictionaries or nested arrays] that way a function can access the object and mutate it, I know it takes more resources but we are not dealing with quantum science,So , I guess we can give a little bit of more ram provided Ram Consumption increases linearly with CPU usage.
Here is an example code:
This program runs 4 threads each of which returns some data for some text L[i] for i in L, the returned data from API is stored in the dictionary,
It may vary from program to program whether it is beneficial or not, for small to medium heavy computation task this object mutation works pretty fast and uses a few percentages of more resources.
忘记所有那些复杂的解决方案。
只需安装合适的软件包,例如:
https://pypi.org/project/thread-with-results/
Forget all that complicated solutions.
Just install a suitable package such as:
https://pypi.org/project/thread-with-results/
我知道这个线程很旧......但我遇到了同样的问题......如果你愿意使用
thread.join()
I know this thread is old.... but I faced the same problem... If you are willing to use
thread.join()
最好的方法...定义一个全局变量,然后在线程函数中更改该变量。没有什么可以传入或检索回来
Best way... Define a global variable, then change the variable in the threaded function. Nothing to pass in or retrieve back