Python进程间通信

发布于 2024-10-16 06:30:31 字数 975 浏览 4 评论 0原文

我有一个 Python 应用程序(带有 GUI,使用 PyQt4),它由用户在多个实例中生成。该应用程序用于执行一些长时间运行的任务(大约几个小时到几天),因此我愿意添加一个额外的“监视”应用程序,该应用程序将执行以下操作:

  • 查找其他应用程序的所有正在运行的进程
  • 获取正在运行的操作的状态(已完成的作业、百分比、错误消息...)
  • 最终会向应用程序发送一些命令,告诉它们暂停、恢复、停止...

适合该作业的一种方法是 RPyC,唯一的问题,它似乎只能通过 TCP 套接字工作,就像我发现的大多数 RPC 库一样。但这导致必须打开多个仅在本地主机上侦听的不需要的套接字,并且必须创建某种端口分配机制以避免两个进程尝试侦听同一端口。然后监视器需要将端口列表写入某处,或者去查找侦听 TCP 端口的进程并尝试确定它们是否是正确应用程序的实例。听起来一团糟。

我目前能想到的管理相互通信的最好方法是拥有一些unix套接字,比方说,在“/var/run/myapp/myapp-.sock”中,然后创建一个模块来完成所有脏东西公开一些方法,例如 listMyApps()getMyApp(pid),第一个返回 pid 列表,第二个返回可用于与该应用程序通信的对象。

现在,我正在寻找实现这一目标的最佳方法.. 事实上,目前还没有任何措施来管理 unix 套接字上的 RPC 吗?对我来说听起来有点奇怪,但我找不到任何合适的东西..

有什么建议吗?

注意:我不想逆转事情(应用程序是单个监控服务器的客户端)以避免监控应用程序崩溃时出现问题,并让我可以自由地创建连接到这些服务器的其他应用程序套接字并发出请求。

注意:安全性不是问题,因为所有这些东西都在私有、封闭和防火墙网络中运行:),此外,请求仅由受信任的用户在本地主机上完成。

I have a Python application (with GUI, using PyQt4) that gets spawned by the user in several instances. The application is used to execute some long running tasks (about some hours to a few days), so I'm willing to add an extra "monitoring" application that will do things like:

  • find all the running processes of the other application
  • get the status of the running operations (jobs done, percentage, error messages, ...)
  • eventually send some commands to the applications telling them to pause, resume, stop, ...

One way that would fit the job is RPyC, the only problem, it seems to work only over TCP sockets, like most of the RPC libraries I found. But this leads to having to open several unneeded sockets listening only on localhost, and having to create some kind of ports allocation mechanism to avoid two processes trying to listen on the same port. And then the monitor needs a list of ports to be written somewhere, or to go and find processes listening on TCP ports and try to figure out whether they are instances of the correct application or ot. Sounds like a mess.

The nicest way of managing the intercommunication I could think of at the moment would be having some unix sockets, let's say, in '/var/run/myapp/myapp-.sock', and then creating a module that does all the dirty stuff exposing some methods like listMyApps() and getMyApp(pid) returning the first a list of pids, the second an object that could be used for communication with that app.

Now, I'm looking at the best way to accomplish that..
For real, there isn't anything out there already done to manage RPC over unix sockets? Sounds a bit strange to me, but I wasn't able to find anything that could fit..

Any suggestions?

NOTE: I don't want to reverse things (applications are client of a single monitoring server) to avoid problems in case of monitoring application crash, and to letting me free to create other applications that connect to these sockets and make requests.

NOTE: Security is not an issue since all this stuff is running in a private, closed and firewalled network :), plus, requests are done on localhost by trusted users only.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

嗼ふ静 2024-10-23 06:30:31

但这导致必须打开
几个不需要的套接字正在监听
仅在本地主机上,并且必须
创建某种端口分配
避免两个进程的机制
尝试监听同一端口

不完全是。首先,这就是大多数进程间通信的工作方式,通过套接字。 TCP 套接字或 UNIX 套接字。这本质上(不完全是)你在管道标准输出等时所做的事情。

您还可以使用操作系统信号。尽管您必须记住,只有每个进程的主线程可以进行信号处理,所以您必须小心不要阻塞它。

但无论采用哪种方式,都几乎无法回避使用套接字连接。

But this leads to having to open
several unneeded sockets listening
only on localhost, and having to
create some kind of ports allocation
mechanism to avoid two processes
trying to listen on the same port

Not quite. First, that's how most interprocess communication works, through sockets. Either TCP sockets or UNIX sockets. That's essentially (not exactly) what you're doing when piping standard out and such.

You could also use OS signals. Although you have to keep in mind that only the main thread of each process can do signal handling, so you have to be careful to not block that.

But any way you do it there's pretty much no getting around using socket connections.

月下凄凉 2024-10-23 06:30:31

另一种方法是使用某种作业控制系统,而不是让应用程序启动自己的进程,而是让它注册要启动的执行任务的作业。然后作业控制系统将监视已启动的作业。作为奖励,通过这种设计,您可以在将来的某个时候将工作分散到多台机器上。

Another way to go is to have some sort of job control system, and instead of having the application start its own processes, have it register jobs to be started that do the task. Then the job control system would be monitoring the jobs that are started. As a bonus, with this design you can spread work out across many machines at some point in the future.

小兔几 2024-10-23 06:30:31

我想我通过使用 dbus-python 找到了灵魂。我能够将它与 Qt4 主循环集成,尽管在网站上他们说只支持 glib(我猜该页面没有更新)。我刚刚做了一个快速测试,它似乎工作得很好(我只是在名为 com.example.myapp. 的总线上使用了一些虚拟函数并列出了实例并从外部客户端连接)。

然后,我可以使用 TCP 上的 RPyC 内容仅用于“主”控制应用程序和“管理器”应用程序之间的通信,“管理器”应用程序的作用非常类似于机器上实例之间的“交换机”。

一些 ascii-art 澄清:

+-------------------+                    
| MASTER APP        |                    +--------------------+
| on my workstation |------ RPyC --------| Server#0 Manager   |
+-------------------+                    +--------------------+
      |                                      | | |
    RPyC                                     | | '-- dbus ---[INSTANCE #0]
      |                                      | '--- dbus ---[INSTANCE #1]
   +--------------------+                    '---- dbus ---[INSTANCE #2]
   | Server#1 Manager   |
   +--------------------+
      | | |
      | | '-- dbus ---[INSTANCE #0]
      | '--- dbus ---[INSTANCE #1]
      '---- dbus ---[INSTANCE #2]

如果有人感兴趣,请告诉我,我也会发布一些代码示例/更多详细信息..

I guess I found a soultion by using dbus-python. I was able to integrate it with the Qt4 mainloop, although on the website they say only glib was supported (I guess that page is not updated). I just did a qick test and it seems to be working nicely (I just esposed some dummy functions on a bus named com.example.myapp.<pid> and listed instances and connected from an external client).

Then, I could use the RPyC stuff over TCP only for the communication between a "master" control application and the "manager" application that acts quite like a "switch" between instances on a machine.

Some ascii-art clarification:

+-------------------+                    
| MASTER APP        |                    +--------------------+
| on my workstation |------ RPyC --------| Server#0 Manager   |
+-------------------+                    +--------------------+
      |                                      | | |
    RPyC                                     | | '-- dbus ---[INSTANCE #0]
      |                                      | '--- dbus ---[INSTANCE #1]
   +--------------------+                    '---- dbus ---[INSTANCE #2]
   | Server#1 Manager   |
   +--------------------+
      | | |
      | | '-- dbus ---[INSTANCE #0]
      | '--- dbus ---[INSTANCE #1]
      '---- dbus ---[INSTANCE #2]

If anyone is interested, just let me know and I'll post some code examples/more details too..

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文