使用线程加快上下文管理器的缓慢初始化

发布于 2025-01-30 05:00:57 字数 581 浏览 2 评论 0原文

我正在从一个我无法轻易更改的Python2.7模块中消费。该模块有一个用于上下文管理器的类的工厂。这非常适合管理对象的寿命,但是初始化此类涉及等待创建基于云的资源,并且可能需要数分钟才能运行。在我程序的范围内,我需要创建两个对象,例如:

import immutable_module

cloud_resource_name1 = "load_ai_model"
cloud_resource_name2 = "create_input_output_model"

with immutable_module.create_remote_cloud_resource(cloud_resource_name1) as a, \
     immutable_module.create_remote_cloud_resource(cloud_resource_name2) as b: 
    result = __do_imporant_thing(a, b)

print(result)

有没有一种方式来调用这两个上下文manager,同时使用线程加快加载时间?

I am consuming from a python2.7 module which I cannot easily change. This module has a factory for a class that is a context manager. This works great for managing the lifetime of the object, but initializing this class involves waiting on creation of a cloud based resource and it can take many minutes to run. Within the scope of my program I need to create two of the objects such as the following:

import immutable_module

cloud_resource_name1 = "load_ai_model"
cloud_resource_name2 = "create_input_output_model"

with immutable_module.create_remote_cloud_resource(cloud_resource_name1) as a, \
     immutable_module.create_remote_cloud_resource(cloud_resource_name2) as b: 
    result = __do_imporant_thing(a, b)

print(result)

Is there a way call these two contextmanagers concurrently using threads to speed up the load time?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

佼人 2025-02-06 05:00:57

在进行了更多研究之后,我找到了解决方案,但这可能不是最好的答案。该解决方案依赖于调用__输入____ exit __手动为上下文管理器手动。

import immutable_module

cloud_resource_name1 = "load_ai_model"
cloud_resource_name2 = "create_input_output_model"

a_context = immutable_module.create_remote_cloud_resource(cloud_resource_name1) 
b_context = immutable_module.create_remote_cloud_resource(cloud_resource_name2)

a = None
b = None

try:
    def __set_a():
      global a
      a = a_context.__enter__()

    def __set_b():
      global b
      b = b_context.__enter__()

    a_thread = Thread(target=__set_a)
    b_thread = Thread(target=__set_b)

    a_thread.start()
    b_thread.start()

    a_thread.join()
    b_thread.join()

    result = __do_imporant_thing(a, b)

finally:
    try:
        if a is not None:
            a_context.__exit__(*sys.exc_info())
    finally:
        if b is not None:
            b_context.__exit__(*sys.exc_info())

print(result)

这确实是肮脏的,根本不是通用的,并且存在异常处理的潜在问题,但有效。

After doing more research I found a solution, but it may not be the best answer. The solution relies on calling __enter__ and __exit__ manually for the contextmanager.

import immutable_module

cloud_resource_name1 = "load_ai_model"
cloud_resource_name2 = "create_input_output_model"

a_context = immutable_module.create_remote_cloud_resource(cloud_resource_name1) 
b_context = immutable_module.create_remote_cloud_resource(cloud_resource_name2)

a = None
b = None

try:
    def __set_a():
      global a
      a = a_context.__enter__()

    def __set_b():
      global b
      b = b_context.__enter__()

    a_thread = Thread(target=__set_a)
    b_thread = Thread(target=__set_b)

    a_thread.start()
    b_thread.start()

    a_thread.join()
    b_thread.join()

    result = __do_imporant_thing(a, b)

finally:
    try:
        if a is not None:
            a_context.__exit__(*sys.exc_info())
    finally:
        if b is not None:
            b_context.__exit__(*sys.exc_info())

print(result)

This is really dirty, not generic at all, and there are potential issues with exception handling, but it works.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文