使用C#进行缓冲池管理
我们需要为使用 C# 开发的应用程序开发某种缓冲区管理。
本质上，应用程序会在设备进入时接收来自设备的消息（短时间内可能会有很多消息）。 我们需要将它们在某种缓冲池中排队，以便我们可以以托管方式处理它们。
我们正在考虑以 256 字节块的形式分配一块内存（所有消息都小于该大小），然后使用缓冲池管理来拥有一个可用于传入消息的可用缓冲区池和一个准备好处理的缓冲区池。
因此流程将是“获取缓冲区”（处理它）“释放缓冲区”或“将其留在池中”。 我们还需要知道缓冲区何时填满。
潜在地，我们还需要一种方法来“查看”缓冲区以查看池中优先级最高的缓冲区是什么，而不是总是获取下一个缓冲区。
.NET 中是否已经支持此功能，或者是否有一些我们可以使用的开源代码？
We need to develop some kind of buffer management for an application we are developing using C#.
Essentially, the application receives messages from devices as and when they come in (there could be many in a short space of time). We need to queue them up in some kind of buffer pool so that we can process them in a managed fashion.
We were thinking of allocating a block of memory in 256 byte chunks (all messages are less than that) and then using buffer pool management to have a pool of available buffers that can be used for incoming messages and a pool of buffers ready to be processed.
So the flow would be "Get a buffer" (process it) "Release buffer" or "Leave it in the pool". We would also need to know when the buffer was filling up.
Potentially, we would also need a way to "peek" into the buffers to see what the highest priority buffer in the pool is rather than always getting the next buffer.
Is there already support for this in .NET or is there some open source code that we could use?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（5）
我正在做类似的事情。 我有来自 MTA 线程的消息需要在 STA 线程上提供服务。
我使用了 BlockingCollection （并行 fx 扩展的一部分），它由多个 STA 线程（可配置，但默认为 xr * 核心数）。 每个线程都尝试从队列中弹出一条消息。 他们要么超时并重试，要么成功弹出消息并为其提供服务。
我已经将它与性能监视器连接起来，以跟踪空闲时间、作业长度、传入消息等，这些可用于调整队列的设置。
您必须实现自定义集合，或者扩展 BC，以实现队列项优先级。
我以这种方式实现它的原因之一是，据我了解，排队理论通常倾向于单线、多服务器（为什么我觉得我会对此感到困惑？）。
I'm doing something similar. I have messages coming in on MTA threads that need to be serviced on STA threads.
I used a BlockingCollection (part of the parallel fx extensions) that is monitored by several STA threads (configurable, but defaults to xr * the number of cores). Each thread tries to pop a message off the queue. They either time out and try again or successfully pop a message off and service it.
I've got it wired with perfmon counters to keep track of idle time, job lengths, incoming messages, etc, which can be used to tweak the queue's settings.
You'd have to implement a custom collection, or perhaps extend BC, to implement queue item priorities.
One of the reasons why I implemented it this way is that, as I understand it, queueing theory generally favors single-line, multiple-servers (why do I feel like I'm going to catch crap about that?).
C# Sharps 内存管理实际上非常好，因此您可以直接分配所需的内容并将其放入队列中，而不需要使用缓冲区池。 一旦你完成了缓冲区的工作，就让垃圾收集器来处理它。
另一种选择（对您的应用程序知之甚少）是在收到消息时对其进行最低程度的处理，并将它们转变为成熟的对象（具有优先级和所有内容），然后您的队列只需通过调查正确的集合即可对它们进行优先级排序属性或方法。
如果您的消息进入速度太快，即使只进行最少的处理，您也可以使用两个队列系统。 第一个队列只是未处理的缓冲区队列，下一个队列是从缓冲区构建的消息对象队列。
我希望这有帮助。
C# sharps memory management is actually quite good, so instead of having a pool of buffers, you could just allocate exactly what you need and stick it into a queue. Once you are done with buffer just let the garbage collector handle it.
One other option (knowing only very little about your application), is to process the messages minimally as you get them, and turn them into full fledged objects (with priorities and all), then your queue could prioritize them just by investigating the correct set of attributes or methods.
If your messages come in too fast even for minimal processing you could have a two queue system. One is just a queue of unprocessed buffers, and the next queue is the queue of message objects built from the buffers.
I hope this helps.
@grieve：网络是本机的，这意味着当使用缓冲区在网络上接收/发送数据时，它们被固定在内存中。 请参阅下面我的评论以进行详细说明。
@grieve: Networking is native, meaning that when buffers are used the receive/send data on the network, they are pinned in memory. see my comments below for elaboration.
为什么不直接接收消息，创建一个 DeviceMessage （由于缺乏更好的名称）对象，然后将该对象放入 Queue 中？ 如果优先级很重要，请实现一个 PriorityQueue 类来自动处理该问题（通过将 DeviceMessage 对象插入队列时按优先级顺序放置）。 看起来更像是一种面向对象的方法，并且会随着时间的推移简化优先级方面的维护。
Why wouldn't you just receive the messages, create a DeviceMessage (for lack of a better name) object, and put that object into a Queue ? If the prioritization is important, implement a PriorityQueue class that handles that automatically (by placing the DeviceMessage objects in priority order as they're inserted into the queue). Seems like a more OO approach, and would simplify maintenance over time with regards to the prioritization.
我知道这是一篇旧文章，但我认为您应该看一下 ILNumerics 项目中实现的内存池。 我认为他们完全满足了您的需要，并且这是一段非常好的代码。
从 http://ilnumerics.net/ 下载代码并查看文件 ILMemoryPool.cs
I know this is an old post, but I think you should take a look at the memory pool implemented in the ILNumerics project. I think they did exactly what you need and it is a very nice piece of code.
Download the code at http://ilnumerics.net/ and take a look at the file ILMemoryPool.cs