PLINQ AsParallel() 优先级较低?

发布于 2024-10-20 15:09:34 字数 230 浏览 2 评论 0 原文

是否可以运行某些 PLINQ AsParallel() - 优先级低于其他查询的查询? (或者某些优先级高于其他) 这可以通过 PLinq 实现吗?还是我必须避免使用 PLINQ 并自己完成所有工作?

编辑/更新:

是否可以在并行执行的方法内部调用

Thread.Sleep(0)

当我想归档较低优先级时, ? 或者这是一个非常糟糕的做法/黑客?

is it possible to run some of my PLINQ AsParallel() - Queries with a lower priority than others?
(Or some with a higher priority than others)
Is this possible with PLinq or will I have to avoid PLINQ and do all the stuff on my own?

EDIT/UPDATE:

Would it be possible to call

Thread.Sleep(0)

inside the parallel executed method when I want to archive a lower priority?
Or is that a very bad practice/hack?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

魔法唧唧 2024-10-27 15:09:34

不幸的是,这在 PLINQ 中不能直接实现。

您可以通过创建自定义 任务计划程序。这将允许您在使用 Parallel.For 或 ForEach 时拥有自定义“优先级”。

然而,PLINQ 并未提供自定义 TaskScheduler 的功能,因为 PLINQ 需要 TaskScheduler 提供非常严格的保证,而且担心暴露这一点会带来很大问题。


编辑/更新:

可以打电话吗

线程睡眠(0)

这会“降低”优先级,但不幸的是,它有自己的问题,特别是与 PLINQ 结合使用时。这可能会导致 ThreadPool 中的线程匮乏,因为您将在 ThreadPool 线程上“休眠”。

此外,这还存在一个根本问题 - PLINQ 是为处理查询而设计的,而不是为处理而设计的。引入逻辑代码来控制流结构确实违背了 PLINQ 背后的理论,并且可能会导致您意想不到的性能影响,尤其是在使用默认分区时。

Unfortunately, this is not directly possible in PLINQ.

You can do it in most of the rest of the Task Parallel Library via creation of a custom TaskScheduler. This would allow you to have custom "priorities" when using Parallel.For or ForEach.

However, the ability to customize the TaskScheduler was not provided with PLINQ, since PLINQ requires very strict guarantees from the TaskScheduler, and the fear was that exposing this would be very problematic.


EDIT/UPDATE:

Would it be possible to call

Thread.Sleep(0)

This would "lower" the priority, but unfortunately, has it's own issues, especially when combined with PLINQ. This will potentially cause thread starvation in the ThreadPool, since you'll be "sleeping" on ThreadPool threads.

In addition, there's a fundamental problem with this - PLINQ is designed and intended to handle queries, and is not designed for processing. Introducing logical code to control the flow structure is really against the theory behind PLINQ, and will likely cause unintended performance repercussions you aren't expecting, especially if you're using the default partitioning.

私野 2024-10-27 15:09:34

AsParallel 是非常高级的 API。如果您想使用 优先级

AsParallel is very high level API. You should really use Threads if you want fine grained control over what is happening using Priority

蓝眼泪 2024-10-27 15:09:34

PLINQ 库不允许配置 TaskScheduler有充分的理由

我们最初没有公开这一点,后来每次提出问题时也没有公开,是因为担心陷入僵局。一些 PLINQ 运算符使用屏障,这样处理查询所涉及的所有任务/分区都需要加入,然后它们才能取得进一步的进展。如果底层调度程序不能保证作为 PLINQ 处理的一部分而排队的所有任务都能够同时运行,例如,如果它是使用固定数量的线程创建的调度程序,那么我们就会面临死锁的风险。

如果您喜欢冒险,可以考虑通过反射进行此配置。下面是一个完全执行此操作的 PLINQ 运算符:

/// <summary>
/// Sets the scheduler that should execute the query.
/// </summary>
/// <remarks>Has been tested on .NET 6 only.</remarks>
public static ParallelQuery<T> WithScheduler<T>(this ParallelQuery<T> source,
    TaskScheduler scheduler)
{
    if (source == null) throw new ArgumentNullException(nameof(source));
    if (scheduler == null) throw new ArgumentNullException(nameof(scheduler));
    const string name1 = "_specifiedSettings";
    FieldInfo fi1 = typeof(ParallelQuery).GetField(name1,
        BindingFlags.NonPublic | BindingFlags.Instance);
    if (fi1 == null) throw new InvalidOperationException($"Field {name1} not found.");
    object querySettings = fi1.GetValue(source);
    if (querySettings == null) throw new InvalidOperationException($"{name1} is null.");
    const string name2 = "TaskScheduler";
    PropertyInfo pi2 = querySettings.GetType().GetProperty(name2,
        BindingFlags.NonPublic | BindingFlags.Instance);
    if (pi2 == null) throw new InvalidOperationException($"Property {name2} not found.");
    pi2.SetValue(querySettings, scheduler);
    fi1.SetValue(source, querySettings); // The QuerySettings is a struct
    return source;
}

使用示例:

TaskScheduler lowPriorityScheduler = new ConcurrentExclusiveSchedulerPair(
    TaskScheduler.Default, Environment.ProcessorCount).ConcurrentScheduler;

var query = source
    .AsParallel()
    .WithScheduler(lowPriorityScheduler)
    //...

您可以使用相同的 lowPriorityScheduler 来调度应以低优先级运行的其他操作,与此查询和其他查询并行运行。所有这些操作组合最多将使用 Environment.ProcessorCount 来自 的线程线程池。超出此限制的所有其他可用线程将专门由非低优先级操作使用。

您还可以查看 这个有点相关的问题。

The PLINQ library does not allow configuring the TaskScheduler, for good reasons:

The reason we didn't make this public initially and then haven't each time the question has come up is due to fear of deadlocks. Some PLINQ operators use barriers, such that all tasks/partitions involved in processing the query need to join before any of them can make further progress. If the underlying scheduler can't guarantee that all of the tasks queued as part of PLINQ's processing will be able to run at the same time, e.g. if it's a scheduler created with a fixed number of threads, then we risk deadlock.

If you feel adventurous you could consider doing this configuration with reflection. Here is a PLINQ operator that does exactly this:

/// <summary>
/// Sets the scheduler that should execute the query.
/// </summary>
/// <remarks>Has been tested on .NET 6 only.</remarks>
public static ParallelQuery<T> WithScheduler<T>(this ParallelQuery<T> source,
    TaskScheduler scheduler)
{
    if (source == null) throw new ArgumentNullException(nameof(source));
    if (scheduler == null) throw new ArgumentNullException(nameof(scheduler));
    const string name1 = "_specifiedSettings";
    FieldInfo fi1 = typeof(ParallelQuery).GetField(name1,
        BindingFlags.NonPublic | BindingFlags.Instance);
    if (fi1 == null) throw new InvalidOperationException(
quot;Field {name1} not found.");
    object querySettings = fi1.GetValue(source);
    if (querySettings == null) throw new InvalidOperationException(
quot;{name1} is null.");
    const string name2 = "TaskScheduler";
    PropertyInfo pi2 = querySettings.GetType().GetProperty(name2,
        BindingFlags.NonPublic | BindingFlags.Instance);
    if (pi2 == null) throw new InvalidOperationException(
quot;Property {name2} not found.");
    pi2.SetValue(querySettings, scheduler);
    fi1.SetValue(source, querySettings); // The QuerySettings is a struct
    return source;
}

Usage example:

TaskScheduler lowPriorityScheduler = new ConcurrentExclusiveSchedulerPair(
    TaskScheduler.Default, Environment.ProcessorCount).ConcurrentScheduler;

var query = source
    .AsParallel()
    .WithScheduler(lowPriorityScheduler)
    //...

You could use the same lowPriorityScheduler to schedule other operations that should run with low priority, in parallel with this and other queries. All these operations combined will use at most Environment.ProcessorCount threads from the ThreadPool. All other threads that are available beyond this limit, will be used exclusively by non-low-priority operations.

You could also take a look at this somewhat related question.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文