如何确保BatchBlock已经完成

发布于 2025-01-13 02:54:46 字数 1448 浏览 3 评论 0原文

我试图了解如何使用 BatchBlock,因为它似乎是我正在从事的项目所需要的。然而,我特别关注一个部分,那就是如何确保批次完成。以下面的代码为例。它的批量大小为 5。五个值被发布到它

using System;
using System.Linq;
using System.Threading.Tasks.Dataflow;

namespace sandbox
{
    class Program
    {

        public static void Main(string[] args)
        {
            BatchBlock<int> batchBlock = new BatchBlock<int>(5);
            ActionBlock<int[]> actionBlock = new ActionBlock<int[]>(OutputAverage);

            batchBlock.LinkTo(actionBlock, new DataflowLinkOptions() { PropagateCompletion = true });

            batchBlock.Post(1);
            batchBlock.Post(2);
            batchBlock.Post(3);
            batchBlock.Post(4);
            batchBlock.Post(5);

            batchBlock.Complete();
            batchBlock.Completion.Wait();
        }

        private static void OutputAverage(int[] values) =>
            Console.WriteLine("The average is: " + values.Average());

    }
}

如果我运行上面的代码,控制台窗口不会输出任何内容。这是因为程序执行在批处理之前就完成了。如果我在最后添加一个超小的 Thread.Sleep(100); ,那么这足以让它完成批处理并将预期的文本输出到控制台。

我还考虑过其他解决方案,包括经典的 Console.ReadLine() 但最终,我需要它可以被调用并运行,直到所有批次正确完成无需有人观看即可进行处理,然后按 Enter 键将其关闭。

我想要使​​用的实际项目是读取一个大型(> 4GB)二进制编码文件并一次处理块以解析记录。 BatchBlock 部分将用于批量解析记录并将其输出到 JSON 文件,与解析并行。目前一切都按预期工作,除了我的测试文件,这些文件很小并且只包含约 400 条记录,程序在批处理线程完成之前退出。

所以最终,我的问题是,我该怎么做才能确保在程序退出之前所有批次都已通过操作处理完成?

I'm trying to get an understanding of using BatchBlock as it seems to be what I'll need for a project I'm working on. However, I'm getting hung up on one part in particular, and that's how to ensure the batch has completed. Take the following code for example. It has a batch size of 5. Five values are posted to it

using System;
using System.Linq;
using System.Threading.Tasks.Dataflow;

namespace sandbox
{
    class Program
    {

        public static void Main(string[] args)
        {
            BatchBlock<int> batchBlock = new BatchBlock<int>(5);
            ActionBlock<int[]> actionBlock = new ActionBlock<int[]>(OutputAverage);

            batchBlock.LinkTo(actionBlock, new DataflowLinkOptions() { PropagateCompletion = true });

            batchBlock.Post(1);
            batchBlock.Post(2);
            batchBlock.Post(3);
            batchBlock.Post(4);
            batchBlock.Post(5);

            batchBlock.Complete();
            batchBlock.Completion.Wait();
        }

        private static void OutputAverage(int[] values) =>
            Console.WriteLine("The average is: " + values.Average());

    }
}

If I run the above code, nothing is output to the console window. This is because the program execution is finishing before the batch has had time to process. If I add a super small Thread.Sleep(100); at the end, then that is enough of a delay for it to complete the batch and output the expected text to the console.

There are other solutions I've though of, including the classic Console.ReadLine() but ultimately, I'm going to need this to be something that can be called and runs until completion with all batches properly processed without the need for someone to watch it and hit enter to close it out.

The actual project I'm wanting to use this is one that reads in a large (>4GB) binary encoded file and processes chunks at a time to parse records. The BatchBlock part will be used to take batches of the records parsed and output them to JSON files in parallel with the parsing. Currently everything is working as expected except in my test files, which are small and only contain ~400 records, the program is exiting before the batch thread finishes.

So ultimately, my question is, what can I do to ensure that all batches have been processed to completion by the action before the program exits?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

你穿错了嫁妆 2025-01-20 02:54:46

您需要等待管道中的最后一个块,而不是第一个块。只需将其替换为:

batchBlock.Completion.Wait();

... 即可:

actionBlock.Completion.Wait();

You need to wait the last block in the pipeline, not the first. Just replace this:

batchBlock.Completion.Wait();

...with this:

actionBlock.Completion.Wait();
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文