在 Scala 中使用并行集合的首选方式是什么?
起初,我假设每个集合类都会收到一个额外的 par
方法,该方法会将集合转换为合适的并行数据结构(例如 map
返回元素类型的最佳集合)在 Scala 2.8 中)。
现在看来,一些集合类支持 par
方法(例如 Array),但其他集合类则支持 toParSeq
、toParIterable
方法(例如 List)。这有点奇怪,因为数组不经常使用或推荐。
这是什么原因呢?让所有集合类都可以使用 par
来做“正确的事情”不是更好吗?
如果我有可以并行处理的数据,我应该使用什么类型? scala.collection 中的特征还是直接实现的类型?
或者我现在应该更喜欢数组,因为它们并行化似乎更便宜?
At first I assumed that every collection class would receive an additional par
method which would convert the collection to a fitting parallel data structure (like map
returns the best collection for the element type in Scala 2.8).
Now it seems that some collection classes support a par
method (e. g. Array) but others have toParSeq
, toParIterable
methods (e. g. List). This is a bit weird, since Array isn't used or recommended that often.
What is the reason for that? Wouldn't it be better to just have a par
available on all collection classes doing the "right thing"?
If I have data which might be processed in parallel, what types should I use? The traits in scala.collection
or the type of the implementation directly?
Or should I prefer Arrays
now, because they seem to be cheaper to parallelize?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
列表不太适合并行处理。原因是要到达列表的末尾,您必须遍历每个元素。因此,您也可以将列表视为迭代器,因此也可以使用更通用的东西,例如
toParIterable
。任何具有快速索引的集合都是并行处理的良好候选者。这包括任何实现 LinearSeqOptimized 的内容,以及树和哈希表。
Array
拥有尽可能快的索引,因此它是一个相当自然的选择。您还可以使用ArrayBuffer
(它有一个par
方法返回ParArray
)之类的东西。Lists aren't that well suited for parallel processing. The reason is that to get to the end of the list, you have to walk through every single element. Thus, you may as well just treat the list as an iterator, and thus may as well just use something more generic like
toParIterable
.Any collection that has a fast index is a good candidate for parallel processing. This includes anything implementing
LinearSeqOptimized
, plus trees and hash tables.Array
has as fast of an index as you can get, so it's a fairly natural choice. You can also use things likeArrayBuffer
(which has apar
method returning aParArray
).