适用于 Java 的 MKL 加速数学库
我查看了 StackOverflow 和 Google 上的相关主题,但运气不佳。我对 Java 也很陌生(我有 C# 和 .NET 背景),所以请耐心等待。 Java 世界中有太多可用的东西,令人眼花缭乱。
我正在开始一个新的 Java-on-Linux 项目,该项目需要一些繁重且高度重复的数值计算(即统计、FFT、线性代数、矩阵等)。因此,最大限度地提高数学运算的性能是一项要求,确保数学正确也是一项要求。因此,我有兴趣寻找一个 Java 库,它可能利用 MKL 等本机加速,并且经过验证(因此商业选择绝对是可能的)。
在 .NET 领域,有高度优化和 MKL 加速的商业数学库,例如 Centerspace NMath 和 Extreme Optimization。 Java中有类似的东西吗?
我发现的大多数 Java 数学库要么似乎没有得到积极维护(例如 Colt),要么似乎没有利用 MKL 或其他本机加速(例如 Apache Commons Math)。
我曾考虑过尝试直接从 Java 中利用 MKL(例如 JNI),但我是 Java 新手(更不用说 Java 和本机库之间的互操作),找到一个已经正确、高效地完成此操作的 Java 库似乎更明智。已证明。
如果我犯了错误或被误导(即使是我提到的任何库)以及我对 Java 产品的无知,我再次表示歉意。对于来自高度商业化的 Microsoft 堆栈的我来说,这是一个全新的世界,因此我很容易在去哪里查找以及我提到的 Java 库方面犯错误。我将非常感谢任何帮助或建议。
I've looked at the related threads on StackOverflow and Googled with not much luck. I'm also very new to Java (I'm coming from a C# and .NET background) so please bear with me. There is so much available in the Java world it's pretty overwhelming.
I'm starting on a new Java-on-Linux project that requires some heavy and highly repetitious numerical calculations (i.e. statistics, FFT, Linear Algebra, Matrices, etc.). So maximizing the performance of the mathematical operations is a requirement, as is ensuring the math is correct. So hence I have an interest in finding a Java library that perhaps leverages native acceleration such as MKL, and is proven (so commercial options are definitely a possibility here).
In the .NET space there are highly optimized and MKL accelerated commercial Mathematical libraries such as Centerspace NMath and Extreme Optimization. Is there anything comparable in Java?
Most of the math libraries I have found for Java either do not seem to be actively maintained (such as Colt) or do not appear to leverage MKL or other native acceleration (such as Apache Commons Math).
I have considered trying to leverage MKL directly from Java myself (e.g. JNI), but me being new to Java (let alone interoperating between Java and native libraries) it seemed smarter finding a Java library that has already done this correctly, efficiently, and is proven.
Again I apologize if I am mistaken or misguided (even in regarding any libraries I've mentioned) and my ignorance of the Java offerings. It's a whole new world for me coming from the heavily commercialized Microsoft stack so I could easily be mistaken on where to look and regarding the Java libraries I've mentioned. I would greatly appreciate any help or advice.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
对于像 FFT(数组上的批量操作)这样的事情,java 中的范围检查可能会降低你的性能(至少最近确实如此)。您可能想要寻找能够优化其索引范围的可证明性的库。
根据 HotSpot 规范
实际上,我会查看 JNI,如果它们各自非常大,则在那里进行批量操作。操作花费的时间越长(即求解大型线性系统或大型 FFT),使用 JNI 就越值得(即使您必须在此处和背面进行 memcpy)。
For things like FFT (bulk operations on arrays), the range check in java might kill your performance (at least recently it did). You probably want to look for libraries which optimize the provability of their index bounds.
According to the The HotSpot spec
I would actually look at JNI, and do your bulk operations there if they are individually very large. The longer the operation takes (i.e. solving a large linear system, or large FFT) the more its worth it to use JNI (even if you have to memcpy there and back).
就我个人而言,我同意您的总体方法,将重量级数学从 Java 卸载到商业级库。
谷歌搜索 Java / MKL 集成 我发现了这个 所以你的建议在技术上是可行的。另一个值得考虑的选择是 NAG 库。我一直使用 MKL,尽管我用 Fortran 编程,所以不存在集成问题。我当然可以推荐他们的质量和性能。例如,我们将 FFTW 的 MKL 版本与我们自己从源代码构建的版本进行了测试。 MKL 实现速度快了一个小整数倍。
如果您担心通过 JNI 调用库的性能,那么您应该计划构建应用程序,以优先进行更少的大型调用,而不是进行更多的小型调用。至于使用 JNI 的困难,我的观点(我已经完成了一些 JNI 编程)是,您在学习如何使用接口方面所付出的最初努力将会得到很好的回报。
我注意到您似乎还没有对可以使用哪些 Java 数学库的建议感到不知所措。和您一样,我会对从网上搜罗的研究质量、低使用率的 Java 库持怀疑态度。
Personally, I agree with your general approach, offloading the heavyweight maths from Java to a commercial-grade library.
Googling around for Java / MKL integration I found this so what you propose is technically possible. Another option to consider would be the NAG libraries. I use the MKL all the time, though I program in Fortran so there are no integration issues. I can certainly recommend their quality and performance. We tested, for instance, the MKL version of FFTW against a version we built from sources ourselves. The MKL implementation was faster by a small integer multiple.
If you have concerns about the performance of calling a library through JNI, then you should plan to structure your application to make fewer larger calls in preference to more smaller ones. As to the difficulties of using JNI, my view (I've done some JNI programming) is that the initial effort you have to make in learning how to use the interface will be well rewarded.
I note that you don't seem to be overwhelmed yet with suggestions of what Java maths libraries you could use. Like you I would be suspicious of research-quality, low-usage Java libraries trawled from the net.
我认为你最好避开它们。我可能是错的,我对它不太熟悉,所以不要从中接受太多,除非其他一些人同意我的观点,但是调用 JNI 的开销相当大,因为它必须到外部JRE 以及执行此操作的所有内容,因此除非您将许多内容组合到一个函数中以立即完成,否则外部库的微小好处将被调用它们的成本所抵消。我会放弃寻找 MKL 库并寻找优化的纯 Java 库。抱歉,我不能说我知道比标准的更好的推荐。
You'd probably be better off avoiding them I think. I could be wrong, it's not a bit I'm too familiar with, so don't take too much from this unless a few others agree with me, but calling up the JNI has quite a large overhead, since it has to go outside of the JRE and everything to do it, so unless you're grouping a lot of things together into a single function to put through at once, the slight benefit of the external library's will be outweighed hugely by the cost of calling them. I'd give up looking for an MKL library and find an optimized pure Java library. I can't say I know of any better than the standard one to recommend though, sorry.