是否有可与 Torque/PBS 配合使用的 DRMAA Java 库？
有人知道 DRMAA-API 的 Java 实现吗？该 API 可以与 PBS/Torque 集群软件一起使用吗？
背后的背景：我想使用符合 DRMAA 的 API 从 Java 向新设置的 Linux 集群提交作业。集群由 PBS/Torque 管理。 Torque 包括用于 Torque/PBS 的 PBS DRMAA 1.0 库，其中包含 DRMA-C 绑定并在 libdrmaa.so 和 .a 二进制文件中提供。我知道Sun 网格引擎包含一个提供Java-DRMAA API 的drmaa.jar。事实上，我选择使用 SGE，但决定先尝试 PBS。
该决定背后的理论是：
“DRMAA 是一个标准，因此 Java API 只需要符合标准的 drmaa-c 绑定。”然而，我找不到这样的“通用 DRMAA-C-java API”，现在假设这个假设是错误的，并且 Java 库是特定于引擎的。
编辑： 我刚刚尝试了 Sun Grid Engine 包中的 drmaa.jar，并尝试将其与 pbs libdrmaa.so 交叉使用。毫不奇怪，失败了（JNI 不满意的链接错误）。
结论：这样不行！经过一番搜索后，我只看到这几个选项：
- Install GridWay ontop of Globus 工具包。安装在 PBS 之上， GridWay 声称提供 DRMAA 爪哇。对我来说看起来太复杂了 环境。
- 废弃 DRMAA，提交给 PBS 调用系统命令qsub、qstat、 来自 Java 等。简单但并非如此 好的。
自己实现一个 drmaa 绑定。 太复杂了...
切换到 Grid Engine。通用电气在我的 意见优于 PBS 关于语言绑定。
我倾向于选择 2. 或 4。有什么建议吗？
Does anybody know a Java implementation of the DRMAA-API that is known to work with PBS/Torque cluster software?
The background behind this: I would like to submit jobs to a newly set-up linux cluster from Java using a DRMAA compliant API. The cluster is managed by PBS/Torque. Torque includes PBS DRMAA 1.0 library for Torque/PBS that contains a DRMA-C binding and provides in libdrmaa.so and .a binaries. I know that Sun grid engine includes a drmaa.jar providing a Java-DRMAA API. In fact I opted to use SGE but it was decided to try PBS first.
The theory behind that decision was:
'DRMAA is a standard and therefore a Java API needs only a standards compliant drmaa-c binding.' However, I couldn't find such 'general DRMAA-C-java API' and now assume that this assumption is wrong and that the Java libraries are engine specific.
Edit:
I just experimented with the drmaa.jar from sun grid engine package and tried to cross-use it with the pbs libdrmaa.so. Not surprisingly, that failed (JNI unsatisfied link error).
Conclusion: It does not work that way! After some search I see only these few options:
- Install GridWay ontop of Globus
toolkit. Installed ontop of PBS,
GridWay claims to provide DRMAA in
Java. Looks far too complex for my
setting. - Scrap DRMAA, submit to PBS by
calling system command qsub, qstat,
etc. from Java. Simple but not so
nice. Implement a drmaa binding myself.
Way too complex...Switch to Grid Engine. GE in my
opinion is superior over PBS
with respect to language bindings.
I tend to prefer option 2. or 4. Any recommendations?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
经过一番搜索后，看来我必须自己写点东西了。似乎还没有最佳答案，但它可以作为对那些尝试相同的人的警告。
提出这些问题的最佳地点可能是 Torque 邮件列表：www.clusterresources.com/resources/mailing-lists.php
首先，为什么您不能只使用任何 DRMAA-Java 库并将其与任何 DRMAA 一起使用 - C 实现是：
DRMAA 描述了资源控制的接口，而不是它是如何实现的。供应商可以使用 DRMAA-C 实现并仅使用这些功能，但他们不是必须这样做。它可以使用引擎中的任何东西。因此，一个重要的信息是：如果您需要某些语言绑定，请确保它们适用于所有所需的语言。
关于提到的选项：
使用 GridWay/Globus 工具包：http://www. gridway.org/doku.php?id=start
优点：Gridway 是一个元调度程序，支持许多资源管理系统（SGE、PBS...）。可能这是目前让 DRMAA 接口与 PBS 配合使用的唯一方法。
缺点：这看起来像是层次和复杂性的膨胀。没有这方面的经验。
使用系统命令 qsub、qstat、qdel。
优点：快速破解
缺点：脏黑客，需要为输出实现解析器，可能不会注意到出现问题，从 stdin/stdout/stderr 传递消息，不可移植
使用 JNI 应该可以为每个 c- 创建一个绑定drmaa.c 中的函数
优点：将提供完整的 drmaa 实现（希望如此）
缺点：涉及编译代码、大量手动包装 C 函数（也许这可以自动化）
切换到另一个网格引擎。也许我们之前就应该进行这样的分析。然而，我们已经有了另一个 Torque 集群，并且有这方面的经验。运营两个将创建更多异构基础设施。
更改来自不同供应商的现有 drmaa 库。不知道这是否可行...我们也会对此进行研究。
After some more searching it looks like I have to write something myself. There seems to be not optimal answer yet, but it can serve as a warning for those attempting the same.
The best place to ask these questions is possibly the Torque mailing list: www.clusterresources.com/resources/mailing-lists.php
First of all, the reason why you cannot just use any DRMAA-Java library and use it with any DRMAA-C implementation is:
DRMAA describes the interface of the resource control, not how it is implemented. The vendor could use a DRMAA-C implementation and use only these functions, but they do not have to. It can use whatever is there in the engine. So one important message is: if you need certain language bindings, make sure they are there for all languages required.
Regarding the options mentioned:
Using GridWay/Globus Toolkit: http://www.gridway.org/doku.php?id=start
Advantage: Gridway is a meta scheduler that supports many resource management systems (SGE, PBS,...). Possibly, the only way to get a DRMAA interface to work with PBS at the moment.
Disadvantage: It seems like an inflation of layers and complexity. Have no experience with that.
Using system commands, qsub, qstat, qdel.
Advantage: quick hack
Disadvantages: dirty hack, need to implement parsers for the output, might not notice if something goes wrong, pass around messages from stdin/stdout/stderr, not portable
Using JNI it should be possible to create a binding for each c-function in drmaa.c
Advantage: would provide a full drmaa implementation (hopefully)
Disadvanteges: involves compiled code, lot of manual wrapping of C-functions (maybe this can be automated)
Switch to another grid engine. Possibly, we should have done this analysis before. However, we already have one other Torque cluster, and there is experience with this. Operating two would create more heterogeneous infrastructure.
Changing an existing drmaa library from a different vendor. No idea if that is feasible... We will look into that too.
我也遇到了这个问题。
这是一个为 PBS/Torque 创建面向对象的 C++ 和 Java DRMAA 绑定的项目。
缺点是您必须能够加载从 C++ 代码创建的库，因此它不是只能作为 .jar 文件分发的“纯 Java”实现，期望最终用户提供“libdrmaa.so” “他们自己。
对于它的价值，这里是：
https://github.com/bryan-lunt/PBSJavaDRMAA/
I also had this problem.
This is a project to create an Object Oriented C++ and Java DRMAA Binding for PBS/Torque.
The downside is that you have to be able to load the library created from the C++ code, so it is not a "pure Java" implementation that can be distributed only as a .jar file, expecting the end user to provide "libdrmaa.so" themselves.
For what it's worth, here it is:
https://github.com/bryan-lunt/PBSJavaDRMAA/
你有没有决定如何处理这个问题？您是否设法使 Java DRMAA 绑定与 Torque/PBS 一起使用？我正在寻找一些在 Torque/PBS 系统上工作的 Java DRMAA 代码，如果您已经完成了艰苦的工作，我很乐意窃取它。
但是，如果您还没有，那么创建一些 Java 绑定应该不会太糟糕，如果没有其他人这样做，我会这样做。几年前，我成功修改了 SGE 的 DRMAA Java 绑定，以便与 Xgrid 的新 DRMAA 实现（现在已经过时，但可能很快就会恢复）配合使用。
我什至写了一篇关于我的经历的简短博客文章（包括一般说明的链接）：
http://edbaskerville.com/2006/07/11/java-bindings-working/
Did you ever decide what to do with this? Did you manage to get Java DRMAA bindings working with Torque/PBS? I'm looking to get some Java DRMAA code working on a Torque/PBS system, and if you've done the hard work already, I'd love to steal it.
However, if you haven't, it shouldn't be too bad to make some Java bindings, and I will do it if nobody else does. Several years ago I successfully modified the DRMAA Java bindings for SGE to work with a new DRMAA implementation for Xgrid (now stale, but perhaps soon to be revived).
I even wrote a brief blog post on my experiences (includes a link to general instructions):
http://edbaskerville.com/2006/07/11/java-bindings-working/