&quot'attributeError:' forkawareLocal'对象没有属性'连接'&quot即使使用process.join()

发布于 2025-02-13 20:13:28 字数 3274 浏览 1 评论 0原文

我正在编写一个脚本,以将许多DNA基因组相互比较,并且我正在尝试使用多处理以使其运行速度更快。所有过程都附加到公共列表,genome_score_avgs

这是我的主要过程:

if __name__ == "__main__":
    start = time.perf_counter()
    with Manager() as manager:
        genome_score_avgs = manager.list()
        processes = [Process(target=compareGenomes, args=(chunk, genome_score_avgs,)) for chunk in divideGenomes('TEST_DIR')]
        for p in processes:
            p.start()
        for p in processes:
            p.join()
    print(genome_score_avgs)
    print(*createTimeline(genome_score_avgs), sep='\n')
    print(f'Finished in {time.perf_counter() - start} seconds')

这是我遇到的错误:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py", line 801, in _callmethod
    conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ayushpal/Coding/PythonStuff/C4DInter/main.py", line 59, in <module>
    print(*createTimeline(genome_score_avgs), sep='\n')
  File "/Users/ayushpal/Coding/PythonStuff/C4DInter/main.py", line 42, in createTimeline
    min_score = min(score_avgs, key=lambda x: x[2])
  File "<string>", line 2, in __getitem__
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py", line 805, in _callmethod
    self._connect()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py", line 792, in _connect
    conn = self._Client(self._token.address, authkey=self._authkey)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 507, in Client
    c = SocketClient(address)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
    s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
<ListProxy object, typeid 'list' at 0x7fc04ea36bb0; '__str__()' failed>

我在类似的堆栈溢出问题中阅读了主要过程比其他过程更早结束,该过程摧毁了共享列表,并且我应该使用> P.Join()对于所有过程。这就是我在做的事情,但是,它仍然给出同样的错误。我应该怎么办?

编辑1: 这是compareGenomes()的代码:

def compareGenomes(genome_pairings, genome_score_avgs):
    scores = []
    for genome1, genome2 in genome_pairings:
        print(genome1, genome2)
        for i, seq in enumerate(genome1.protein_seqs):
            for j, seq2 in enumerate(genome2.protein_seqs[i::]):
                alignment = align.globalxx(seq, seq2)
                scores.append(alignment)
        top_scores = []

        for i in range(len(genome1.protein_seqs)):
            top_scores.append(max(scores, key=lambda x: x[0][2] / len(x[0][1])))
            scores.remove(max(scores, key=lambda x: x[0][2] / len(x[0][1])))

        avg_score = sum([i[0][2] / len(i[0][1]) for i in top_scores]) / len(top_scores)

        with open(f'alignments/{genome1.name}x{genome2.name}.txt', 'a') as file:
            file.writelines([format_alignment(*i[0]) for i in top_scores])
        genome_score_avgs.append((genome1, genome2, avg_score))

I'm writing a script for comparing many DNA genomes with each other, and I'm trying to use multiprocessing to have it run faster. All the processes are appending to a common list, genome_score_avgs.

This is my main process:

if __name__ == "__main__":
    start = time.perf_counter()
    with Manager() as manager:
        genome_score_avgs = manager.list()
        processes = [Process(target=compareGenomes, args=(chunk, genome_score_avgs,)) for chunk in divideGenomes('TEST_DIR')]
        for p in processes:
            p.start()
        for p in processes:
            p.join()
    print(genome_score_avgs)
    print(*createTimeline(genome_score_avgs), sep='\n')
    print(f'Finished in {time.perf_counter() - start} seconds')

This is the error that I'm getting:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py", line 801, in _callmethod
    conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ayushpal/Coding/PythonStuff/C4DInter/main.py", line 59, in <module>
    print(*createTimeline(genome_score_avgs), sep='\n')
  File "/Users/ayushpal/Coding/PythonStuff/C4DInter/main.py", line 42, in createTimeline
    min_score = min(score_avgs, key=lambda x: x[2])
  File "<string>", line 2, in __getitem__
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py", line 805, in _callmethod
    self._connect()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py", line 792, in _connect
    conn = self._Client(self._token.address, authkey=self._authkey)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 507, in Client
    c = SocketClient(address)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
    s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
<ListProxy object, typeid 'list' at 0x7fc04ea36bb0; '__str__()' failed>

I read in a similar Stack Overflow question that the main process is ending earlier than the other processes, which destroys the shared list, and that I should use p.join() for all the processes. This is what I'm doing , however, it's still giving the same error. What should I do?

EDIT 1:
this is the code for CompareGenomes():

def compareGenomes(genome_pairings, genome_score_avgs):
    scores = []
    for genome1, genome2 in genome_pairings:
        print(genome1, genome2)
        for i, seq in enumerate(genome1.protein_seqs):
            for j, seq2 in enumerate(genome2.protein_seqs[i::]):
                alignment = align.globalxx(seq, seq2)
                scores.append(alignment)
        top_scores = []

        for i in range(len(genome1.protein_seqs)):
            top_scores.append(max(scores, key=lambda x: x[0][2] / len(x[0][1])))
            scores.remove(max(scores, key=lambda x: x[0][2] / len(x[0][1])))

        avg_score = sum([i[0][2] / len(i[0][1]) for i in top_scores]) / len(top_scores)

        with open(f'alignments/{genome1.name}x{genome2.name}.txt', 'a') as file:
            file.writelines([format_alignment(*i[0]) for i in top_scores])
        genome_score_avgs.append((genome1, genome2, avg_score))

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

酒儿 2025-02-20 20:13:28

发生错误是因为您关闭经理后使用了托管列表。一旦发生这种情况,经理产生的过程也关闭了,因此您的托管列表将不再起作用。您需要使用块中的内的列表,如下所示:

if __name__ == "__main__":
    start = time.perf_counter()
    with Manager() as manager:
        genome_score_avgs = manager.list()
        processes = [Process(target=compareGenomes, args=(chunk, genome_score_avgs,)) for chunk in divideGenomes('TEST_DIR')]
        for p in processes:
            p.start()
        for p in processes:
            p.join()
        print(genome_score_avgs)
        print(*createTimeline(genome_score_avgs), sep='\n')
        
    print(f'Finished in {time.perf_counter() - start} seconds')

The error is happening because you are using the managed list after you have closed the manager. Once that happens, the process that the manager spawns is closed as well, and therefore your managed list will no longer work. You need to use the list inside the with block like below:

if __name__ == "__main__":
    start = time.perf_counter()
    with Manager() as manager:
        genome_score_avgs = manager.list()
        processes = [Process(target=compareGenomes, args=(chunk, genome_score_avgs,)) for chunk in divideGenomes('TEST_DIR')]
        for p in processes:
            p.start()
        for p in processes:
            p.join()
        print(genome_score_avgs)
        print(*createTimeline(genome_score_avgs), sep='\n')
        
    print(f'Finished in {time.perf_counter() - start} seconds')
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文