同时更新多处理器函数（ray）内部的进度栏

发布于 2025-02-09 05:35:25 字数 1032 浏览 2 评论 0原文

我正在编写一个使用射线软件包进行多处理编程的程序。在程序中，有一个函数同时称为5次。在执行过程中，我想使用pyqt5 qprogressbar显示一个进度栏，以指示完成了多少工作。我的想法是让函数的每个执行都会更新进度栏20％。因此，我按以下内容编写了代码：

 running_tasks = [myFunction.remote(x,y,z,self.progressBar,QApplication) for x in myList]
 Results = list(ray.get(running_tasks))

在myfunction内部，有一条线可以更新已发送的进度栏，如以下内容：

QApplication.processEvents()
progressBar.setValue(progressBar.Value()+20)

但是，当我运行代码时，我会收到以下错误：

TypeError：无法序列化参数＆lt; pyqt5.qtwidgets.qprogressbar object in 0x000001b787a36b80＆gt;对于任务或演员myfile.myfunction。查看 https://docs.ray.io ray.io/en/en/en/master/master/serialization.html#html#故障排除更多信息。

我通过Internet搜索（URL返回404），我知道此错误是因为Ray中的多处理在处理器之间没有共享内存，并且发送类属性（例如Self.Prgressbar）会导致每个处理器具有其属性拥有仅在本地修改它的副本。我还尝试使用多处理软件包而不是射线，但它会引发腌制错误，我认为这是由于相同的原因。那么，谁能确认我是否对吗？还是提供有关错误的进一步解释？另外，如果多处理在处理器之间没有共享内存？

原文

I'm writing a program that uses ray package for multiprocessing programming. In the program, there is a function that would be called 5 times at the same time. During the execution, I want to show a progress bar using PyQT5 QprogressBar to indicate how much work is done. My idea is to let every execution of the function updates the progress bar by 20%. So I wrote the code like the following:

 running_tasks = [myFunction.remote(x,y,z,self.progressBar,QApplication) for x in myList]
 Results = list(ray.get(running_tasks))

Inside myFunction, there is a line to update the sent progress bar as the following:

QApplication.processEvents()
progressBar.setValue(progressBar.Value()+20)

But, when I run the code, I got the following error:

TypeError: Could not serialize the argument
<PyQt5.QtWidgets.QProgressBar object at 0x000001B787A36B80> for a task
or actor myFile.myFunction. Check
https://docs.ray.io/en/master/serialization.html#troubleshooting for
more information.

I searched through the internet (The URL returns 404) and I understand that this error is because multiprocessing in ray doesn't have shared memory between the processors, and sending a class attribute (like self.prgressBar) will lead each processor to have its own copy where it will modify it locally only. I also tried using the multiprocessing package instead of ray but it throws a pickling error, and I assume it is due to the same reason. So, Can anyone confirm if I'm right? or provide a further explanation about the error?
Also, how can I achieve my requirement in multiprocessing (i.e. updating the same progress bar simultaneously) If multiprocessing doesn't have shared memory between the processors?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

∞梦里开花 2025-02-16 05:35:25

我不熟悉ray，但是您可以在多处理库中使用多处理（）。

队列完全按照命名，您可以在其中放置数据以读取其他多进程的队列。就我而言，我通常用命令（键）将字典在队列中放置，该命令（value）如何处理。

在一个多处理中，您将执行queue.put（），而在另一个多”可以执行queue.get（）。如果您想通过一个方向。在下面的示例中，我模仿您可能要做的事情。

我通常使用QTimer检查队列中是否有任何数据，但是您也可以在何时通过调用方法来检查。

from multiprocessing import Process, Queue
myQueue = Queue()

class FirstProcess():
    ...

    def update_progress_percentage(self, percentage):
        self.progresss_percentage = percentage

    def send_data_to_other_process(self):
        myQueue.put({"UpdateProgress":self.progresss_percentage})

class SecondProcess():
    ...
    
    def get_data_from_other_process(self):
        while not myQueue.empty():
            queue_dict = myQueue.get()
            for key in queue_dict :
                if key == "UpdateProgress":
                    percentage = queue_dict["UpdateProgress"]
                    progressBar.setValue(percentage)

I am unfamiliar with ray, but you can do this in the multiprocessing library using the multiprocessing.Queue().

The Queue is exactly as it's named, a queue where you can put data for other multiprocesses to read. In my case I usually put a dictionary in the Queue with a Command (Key) and what to do with that command (Value).

In one multiprocess you will do Queue.put() and in the other you can do Queue.get(). If you want to pass in one direction. In the example below I emulate what you may be looking to do.

I usually use a QTimer to check if there is any data in the queue, but you can also check whenever you feel like by calling a method to do so.

from multiprocessing import Process, Queue
myQueue = Queue()

class FirstProcess():
    ...

    def update_progress_percentage(self, percentage):
        self.progresss_percentage = percentage

    def send_data_to_other_process(self):
        myQueue.put({"UpdateProgress":self.progresss_percentage})

class SecondProcess():
    ...
    
    def get_data_from_other_process(self):
        while not myQueue.empty():
            queue_dict = myQueue.get()
            for key in queue_dict :
                if key == "UpdateProgress":
                    percentage = queue_dict["UpdateProgress"]
                    progressBar.setValue(percentage)

回复收藏 0 原文

~没有更多了~