Inline::Java 与 Parallel::ForkManager 冲突

发布于 2024-12-09 19:37:26 字数 925 浏览 1 评论 0原文

我在同时调用 Parallel::ForkManagerInline::Java 时遇到问题。具体来说,如果我使用 JNI => 调用 Inline::Java , 1 选项(我必须这样做),那么 fork 进程不会返回到父进程。以下是代码:

use Parallel::ForkManager;

##### Calling Inline::Java #####
use Inline Java => <<END, JNI => 1;

END
###### End of Inline::Java #####

my $pm = Parallel::ForkManager->new(2);
for my $i (0..1) {
    $pm->start and next;
    print "Inside process $i\n";
    $pm->finish;
}
$pm->wait_all_children;
print "Back to Parent.\n";

如果我运行这个程序,它会进入子进程,但永远不会返回父进程。如果我删除注释之间的 3 行,它就可以正常运行。如果我更改 JNI => 1 到 JNI => 0(并不是说我可以出于我的目的更改该参数),然后出现错误消息 Lost connection with Java virtual machine at /usr/lib64/perl5/site_perl/5.8.8 /x86_64-linux-thread-multi/Inline/Java.pm 第 975 行

有谁知道如何解决冲突?我还必须在并行过程之前调用 Inline::Java,因此在并行完成后使用 require 不是一个选项。谢谢!

I am having problem with calling both Parallel::ForkManager and Inline::Java at the same time. Specifically, if I call the Inline::Java with the JNI => 1 option (which I have to), then the fork process doesn't come back to the parent. Here are the codes:

use Parallel::ForkManager;

##### Calling Inline::Java #####
use Inline Java => <<END, JNI => 1;

END
###### End of Inline::Java #####

my $pm = Parallel::ForkManager->new(2);
for my $i (0..1) {
    $pm->start and next;
    print "Inside process $i\n";
    $pm->finish;
}
$pm->wait_all_children;
print "Back to Parent.\n";

If I run this program, it goes into the child processes but never comes back to the parent. If I remove the 3 lines between the comments, it runs fine. If I change the JNI => 1 to JNI => 0 (not that I'm allowed to change that parameter for my purpose), then there is an error message of Lost connection with Java virtual machine at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/Inline/Java.pm line 975.

Does anyone have a clue how to resolve the conflict? I also have to call the Inline::Java before the parallel process so using require after parallel is done is not an option. Thx!

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

—━☆沉默づ 2024-12-16 19:37:26

每个孩子都通过同一个套接字进行通话,这会导致虚拟机接收到乱码。

您需要延迟与虚拟机的连接,以便在子级而不是父级中完成。

您可以将所有与 Inline::Java 相关的内容移动到另一个模块中,然后在 start 之后使用 require Child; (而不是 use Child;)。

如果您需要在启动子进程之前使用 Inline::Java,请在不同的进程中执行此操作。

Every child is talking over the same socket, which leads to the VM receiving jibberish.

You need to delay making a connection to the VM so that it's done in the children instead of it the parent.

You could move all thing Inline::Java-related into another module, then use require Child; (not use Child;) after start.

If you need to use Inline::Java before launching the child, do it in a different process.

眼泪淡了忧伤 2024-12-16 19:37:26

Inline::Java 中使用 fork 将会是一个问题。您的 Perl 脚本需要与 JVM 保持 TCP 连接。当您分叉一个新进程时,用于与 JVM 通信的相同文件描述符将传递给子进程,因此父进程和所有子进程都使用相同的套接字。那是行不通的。您需要重新设计您的应用程序。

一种可能性(您已经排除了这种可能性)是延迟启动 JVM,直到分叉之后,在每个子进程中启动一个新的 JVM。

另一种方法是忘记从 Perl 派生并利用 Java 的高级线程模型来进行并行化。设计您的 Java 代码以在新线程中执行其任务,并从 Perl 启动新线程:

my $java = ... entry point to JVM ...

for my $n (1 .. $num_threads) {
    $java->startNewThread(@inputs)
}
$java->waitForThreadsToFinish();
$result = $java->getResults();

Perl 也有自己的线程模型(请参阅 线程threads::shared)。我怀疑 Perl 的线程能否解决这个问题,但它仍然值得一试。


更新: Inline::Java docs 是使用共享JVM。使用选项 SHARED_JVM => 调用 Inline::Java 1,当新的子进程启动时,从子进程中调用 Inline::Java::reconnect_JVM() 来建立新的连接。这种方法的缺点是

  1. 它在程序结束后使 JVM 保持活动状态,因此您必须记住杀死 JVM,
  2. 因为它与选项 JNI =>; 不兼容。 1,这可能会破坏OP的交易。

Using forks with Inline::Java is going to be a problem. Your perl script needs to maintain a TCP connection with the JVM. When you fork a new process, the same file descriptors for communicating with the JVM are passed to the child process so the parent and all the child processes are using the same sockets. That won't work. You need to redesign your application.

One possibility (which you have already discounted) is to delay starting the JVM until after a fork, starting a new JVM in each child process.

Another approach is to forget about forking from Perl and to leverage Java's superior threading model to do parallelization. Design your Java code to perform its tasks in new threads, and start new threads from Perl:

my $java = ... entry point to JVM ...

for my $n (1 .. $num_threads) {
    $java->startNewThread(@inputs)
}
$java->waitForThreadsToFinish();
$result = $java->getResults();

Perl also has its own threading model (see threads and threads::shared). I doubt that Perl's threads will work for this problem, but it still might be worth a try.


Update: another possibility that is mentioned in the Inline::Java docs is to use a shared JVM. Invoke Inline::Java with the option SHARED_JVM => 1, and when a new child process starts, call Inline::Java::reconnect_JVM() from the child to make a fresh connection. The downsides of this approach are

  1. it keeps the JVM active after the program ends, so you have to remember to kill the JVM
  2. it is incompatible with the option JNI => 1, which might be a dealbreaker to the OP.
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文