Clojure 矩阵表示
Clojure 中矩阵的良好表示形式是什么?我对处理浮点数的密集矩阵感兴趣。 “列表的列表”表示法浮现在脑海中,但是还有更好的吗?
良好表示的一些标准包括:
- 效率:它们不会用于持续处理巨大的数据集,但我不想花费数小时计算结果,而如果有更好的设计,这些结果本来可以在几分钟内完成。
- Java 互操作性:如果能够轻松地在两种语言之间来回传递数据,那就太好了。
- 轻松并行化:如果我只需用
pmap
替换map
就可以使用所有可用的内核,那就太好了。 - 适合使用
reduce
:看起来我正在做的许多计算都可以很好地使用reduce
。 - 在矩阵行中表示图像扫描线的能力:并不是非常重要,但如果有的话就太好了。
有什么想法吗?
What is a good representation for matrices in Clojure? I'm interested in dealing with dense matrices of floating point numbers. The "list of lists" representation springs to mind, but is there something better?
Some criteria for a good representation include:
- Efficiency: They won't be used for constant processing of huge data sets, but I don't want to spend hours calculating results that could have been done in minutes with a better design.
- Java Interoperability: It would be nice to pass the data back and forth between the two languages easily.
- Easy Parallelization: If I can use all the cores available simply by replacing
map
withpmap
, that would be nice. - Amenable to the use of
reduce
: It seems like lots of the calculations I'm doing work very well withreduce
. - Ability to Represent Image Scan Lines in Matrix Rows: Not really very important, but would be nice to have.
Any thoughts?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(8)
Incanter 提供了一些 Parallel Colt,包括看起来相当不错的快速并行密集矩阵实现,与 Clojure 基于 seq 的库接口。我没用过,但它应该是你要找的。
示例。
Incanter supplies a wrapper around some of Parallel Colt, including what looks to be a pretty decent implementation of fast, parallelized dense matrices that interface with Clojure's seq-based libraries. I haven't used it, but it should be what you're looking for.
Example.
我正在编写一个矩阵库,包装 jblas ,暂时称为 Clatrix。它缺少很多我仍然想添加的功能,但它已经提供了您可能正在寻找的大部分功能。看一下,http://github.com/tel/clatrix。
I'm writing a matrix library wrapping jblas called, tentatively, Clatrix. It's missing a lot of features I still want to add, but it's got most of what you might be looking for. Take a look, http://github.com/tel/clatrix.
在这里查看 core.matrix 提案 + 实验性实现:
https://github.com/mikera/matrix-api
在撰写本文时还处于早期阶段,但值得关注。
Check out the core.matrix proposal + experimental implementation here:
https://github.com/mikera/matrix-api
Very early days at time of writing, but worth keeping an eye on.
我目前在 cryptovide 中使用列表列表方法,因为它对于此应用程序非常重要让事情变得懒惰。我也在考虑切换到更有效的方法,只要它至少保持外部表示的惰性。
I am presently using the list of lists approach in cryptovide because its very important for this application to keep things lazy. I am also considering switching to a more efficient approach as long as it kept at least the outward representation lazy.
Rich Hickey 的 Clojure 是一个基于 JVM 的 Lisp,它用 32 路树表示 PercientVector(不是 PercientList)。
如果您想编写自己的矩阵类型,我将使用 PersistentVector
否则最好的选择是使用 Parallel Colt 和 Incanter。
Rich Hickey’s Clojure is a JVM-based Lisp that represents PersistentVector (not a PersistentList) with a 32-way tree.
If you would like to write your own matrix Type i would use PersistentVector
otherwise the best choice is to use Parallel Colt with Incanter.
我最近编写了一些需要矩阵数学的代码,最初我使用了向量的向量、映射和向量。减少,但当我返回时发现生成的代码很难理解(我对 Clojure 很陌生)。 Incanter 使相同的代码变得非常简洁、易于理解(标准矩阵运算)并且速度更快。
I recently wrote some code that needed matrix maths, and initially I used vector-of-vectors, map & reduce, but found the resulting code hard to understand when I returned to it (I'm new to Clojure mind). Incanter made the same code very terse, easy to understand (standard matrix ops) and much much faster.
8 年过去了,答案可能需要更新。快速谷歌搜索显示,如果您需要与 Clojure core.matrix API 兼容,您可以使用 core.matrix 本身或其他实现,例如 vectorz-clj。
此外,我发现 Neanderthal 针对 GPU 进行了优化
the answers may need to be updated as 8 years passed. A quick google search shows that if you need to be compatible with Clojure core.matrix API, you can use core.matrix itself or other implementations such as vectorz-clj.
In addition I found Neanderthal which is optimized for GPU
我不是专家,但无论如何,这是我的观点 :)
列表列表可能是表示矩阵的最自然的 Clojure 习惯用法。这种结构也非常适合映射/归约类型的操作。 Clojure 在处理序列方面也非常高效——可能比大多数替代方案更好。
我不敢保证这一点,但我想我已经看到 Clojure 在我编写的程序上努力运行 3 个或全部 4 个 CPU,这些程序在风格上具有功能性,但没有做出任何并行尝试。我怀疑编译器正在自行寻找一些并行处理的机会。
我认为 Clojure 创建的序列类型将像 Java 中的列表一样工作,或者至少是可迭代的。这可能足以满足您的需求,但如果您尝试在 Java 中将这些结构视为可修改,则可能会遇到问题。
最好按顺序访问列表。如果您打算在矩阵中进行多次跳转,那么从性能角度来看,向量的向量可能更适合您。我怀疑使用 nth 函数会更好。
作为一名前 C 程序员,我简单地认为您可以将矩阵实现为一维结构(即直序列或更好的向量),并进行自己的索引计算以找到正确的元素。您可以使用 partition 函数来逐步执行它...好吧,这可以工作,但我怀疑有很好的理由不这样做。
I'm no expert, but here's my opinion anyway :)
list-of-lists is probably the most natural Clojure idiom for representing matrices. This structure also lends itself nicely to map/reduce kinds of operations. Clojure is also pretty efficient at handling sequences - probably better than most alternatives.
I can't swear to this, but I think I've seen Clojure working 3 or all 4 of my CPUs hard on programs I wrote that were functional in style but didn't make any attempt to be parallel. I suspect the compiler is finding some opportunities for parallel processing on its own.
I think the sequence types created by Clojure will work as Lists in Java, or at least be Iterable. That's probably good enough for what you want, though you may run into problems if you try to treat those structures as modifiable in Java.
Lists are best accessed sequentially. If you're planning to jump around a lot in the matrix, a vector-of-vectors may suit you a little better, performance-wise. I suspect that beats using the nth function.
As a former C programmer, I briefly considered that you could implement your matrix as a one-dimensional structure (i.e. a straight sequence or better a vector), and do your own index computations to find the right element. You could use the partition function to step through it... well, that could be made to work but I suspect there are very good reasons not to.