PLinq 本质上比 System.Threading.Tasks.Parallel.ForEach 更快

发布于 2024-10-20 20:40:44 字数 1405 浏览 0 评论 0原文

总结:我从 System.Threading.Tasks.Parallel.ForEach 和并发数据结构更改为简单的 plinq(并行 Linq)查询。速度惊人

那么 plinq 本质上比 Parallel.ForEach 更快吗?或者它是否特定于任务。

// Original Code
// concurrent dictionary to store results
var resultDict = new ConcurrentDictionary<string, MyResultType>();

Parallel.ForEach(items, item =>
        {
            resultDict.TryAdd(item.Name, PerformWork(source));
        });


// new code

var results =
            items
            .AsParallel()
            .Select(item => new { item.Name, queryResult = PerformWork(item) })
            .ToDictionary(kv => kv.SourceName, kv => kv.queryResult);

注释: 现在,每个任务 (PerformWork) 的运行时间在 0 到 200 毫秒之间。我以前花了更长的时间才优化它。这就是我首先使用 Tasks.Parallel 库的原因。因此,我将总时间从 2 秒缩短到约 100-200 毫秒,执行大致相同的工作,只是使用不同的方法。 (哇 linq 和 plinq 太棒了!)

问题

  1. 速度是否因使用 plinq 与 Parallel.ForEach 而提高?
  2. 是否只是简单地删除并发数据结构(ConcurrentDictionary)? (因为它不需要同步线程)。
  3. 基于此相关问题的答案

虽然 PLINQ 主要基于没有副作用的函数式编程风格,但副作用正是 TPL 的目的。如果您想真正并行工作而不只是并行搜索/选择内容,则可以使用 TPL。

我是否可以假设因为我的模式基本上是功能性的(让输入产生新的输出而不发生突变),所以 plinq 是正确的技术?

我正在寻找我的假设是否正确的验证,或者是否有迹象表明我遗漏了某些内容。

Summary: I changed from System.Threading.Tasks.Parallel.ForEach and Concurrent Data structure to a simple plinq (Parallel Linq) query. The speed up was amazing.

So is plinq inherently faster than Parallel.ForEach? Or is it specific to the task.

// Original Code
// concurrent dictionary to store results
var resultDict = new ConcurrentDictionary<string, MyResultType>();

Parallel.ForEach(items, item =>
        {
            resultDict.TryAdd(item.Name, PerformWork(source));
        });


// new code

var results =
            items
            .AsParallel()
            .Select(item => new { item.Name, queryResult = PerformWork(item) })
            .ToDictionary(kv => kv.SourceName, kv => kv.queryResult);

Notes:
Each task (PerformWork) now runs between 0 and 200 ms. It used to take longer before I optimized it. That's why I was using the Tasks.Parallel library in the fist place. So I went from 2 seconds total time to ~100-200 ms total time, performing roughly the same work, just with different methods. (Wow linq and plinq are awesome!)

Questions:

  1. Is the speed up due to using plinq vs Parallel.ForEach?
  2. Is it instead simply the removal of the concurrent data structure (ConcurrentDictionary)? (Because it doesn't need to synchronize threads).
  3. Based on the answer from this related question

Whereas PLINQ is largely based on a functional style of programming with no side-effects, side-effects are precisely what the TPL is for. If you want to actually do work in parallel as opposed to just searching/selecting things in parallel, you use the TPL.

Can I assume that because my pattern is basically functional (giving inputs produce new outputs without mutation), that plinq is the correct technology to use?

I'm looking for validation that my assumptions are correct, or an indication that I'm missing something.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

╄→承喏 2024-10-27 20:40:44

无法使用这 2 个代码示例在 Parallel.ForEach 和 PLINQ 之间进行明确的比较。代码示例实在是太不同了。

我首先想到的是第一个示例使用 ConcurrentDictionary,第二个示例使用 Dictionary。这两种类型具有截然不同的用途和性能特征。为了准确比较两种技术,您需要在此处保持类型一致。

It's not possible to use these 2 code samples to do a definitive comparison between Parallel.ForEach and PLINQ. The code samples are simply too different.

The first item that jumps out at me is the first sample uses ConcurrentDictionary and the second uses Dictionary. These two types have very different uses and performance characteristics. In order to get an accurate comparison between the two technologies you need to be consistent here with the types.

辞慾 2024-10-27 20:40:44

Based on the limited information you've provided in your sample (I asked for more details in a comment on the OP), I'm guessing sure you're seeing differences due to the partitioning algorithm that is used. You should read up on Chunk Partitioning vs. Range Partitioning in this blog post where he discusses how they differ and for which types of work they might be best suited for. Highly recommend you read that blog article as well as this one which goes into a little more detail on those two types along with two other types of partitioning that can be used, though not applicable to your sample, as well as giving some visual aids to better understand the partitioning. Finally, here's yet another blog post that discusses work partitioning and how it can affect you when the default partitioning algorithm doesn't make sense for your particular workload. That post actually refers to a great program that helps you visualize the partitioners at work that's part of a set of parallel samples from the PFX team.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文