如何使用__NAME__ ==' __ main __&#__'如何从另一个文件中执行文件。用于使用多处理

发布于 2025-02-01 14:25:38 字数 1922 浏览 2 评论 0原文

我目前正在尝试并行化一件与Pytorch训练神经网络的代码。为此,我想使用多处理库。我现在想使用以下结构的代码集合:

test1.py文件:

import runpy

runpy.run_module('test2', run_name= '__main__')

test2.py文件:

import multiprocessing as mp
from test3 import function

a = 1
b = 2

print(__name__)

def main():
    print(__name__)
    if __name__ == '__main__':
        ctx = mp.get_context('spawn')
        index = [1,2,3,4]
    
        for i in index:
            processes = []
            process = ctx.Process(target=function, args=(a,b))
            process.start()
            processes.append(process)
        for process in processes:
            process.join()
        
    main()

test3.py文件:

def function(a,b):
print(a+b)

用于执行test2.py明确所有内容都按预期进行,而一个可以:

__main__
__main__
__mp_main__
__mp_main__
3
__mp_main__
__mp_main__
3
__mp_main__
__mp_main__
3
__mp_main__
__mp_main__
3

执行文件test1.py __名称__属性不会为子解释器打开一个过程而不会更改一个错误:

RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

如果__ -name __ =='__ main __': line,则实际上应由解决。 ,但不是在这种情况下。我现在的问题是:

  1. 为什么使用Runpy不起作用,更重要的是:
  2. 如何解决此问题仍在另一个文件中执行test2.py文件? runpy是问题所在,另一种方法可以更好地执行文件吗?

抱歉,如果问题非常小,但我仍然是初学者。

如果重要的话,我正在使用Spyder致力于Ubuntu。

哦,如果有人问为什么我在Ubuntu上使用Spawn:不幸的是,Pytorch Pachage只支持Spawn上下文。

I am currently trying to parallelize a piece of code that trains a neural network with pytorch. For that I want to use the multiprocessing library. I now want to use a code ensemble of the following structure:

test1.py file:

import runpy

runpy.run_module('test2', run_name= '__main__')

test2.py file:

import multiprocessing as mp
from test3 import function

a = 1
b = 2

print(__name__)

def main():
    print(__name__)
    if __name__ == '__main__':
        ctx = mp.get_context('spawn')
        index = [1,2,3,4]
    
        for i in index:
            processes = []
            process = ctx.Process(target=function, args=(a,b))
            process.start()
            processes.append(process)
        for process in processes:
            process.join()
        
    main()

test3.py file:

def function(a,b):
print(a+b)

For executing test2.py explicitly everything goes as expected and one gets:

__main__
__main__
__mp_main__
__mp_main__
3
__mp_main__
__mp_main__
3
__mp_main__
__mp_main__
3
__mp_main__
__mp_main__
3

Whereas for executing file test1.py the __name__ attribute does not get changed for the child interpreter opening a process and one gets an error like:

RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

Which should actually be resolved by the if __name__ == '__main__': line, but is not in that case. My questions now are:

  1. Why does that not work woth using runpy, and more importantly:
  2. How can I resolve this still executing the test2.py file from another file? Is runpy the problem, and another way to execute the file better?

Sorry, if the question is pretty trivial, but I am still a beginner.

If important, I am working on ubuntu using spyder.

Oh, and if anybody asks why I use spawn on ubuntu: The pytorch pachage unfortunately only supports the spawn context.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文