缓存网络流
我的 C# 程序通过多个套接字接收大量数据，并且似乎出现 Windows 内核上的 TCP 缓存过载的问题。
这可以通过服务器 tcpdump 的以下结果看出：
18:02:18.963370 IP xxxx> yyyy.zzz: . ack 10017011 获胜 0
您会使用哪种数据结构/策略来缓存读取的数据以避免 C# 过载？
我目前正在为单独线程中的每个套接字使用委托，该线程在每个读取记录后调用，但这似乎是一个问题，因此我可能需要将其拆分为两个线程。
或者我应该考虑调整 Windows 端的 TCP 参数？我使用的是Windows 7。
有关于内核优化的书籍吗？
令我大吃一惊的是，尽管整机负载约为 20-25%，最高负载核心负载约为 50-60%，但程序却变慢了。
My C# program is receiving a lot of data via multiple sockets and it appears that I have problems that the TCP cache on the Windows Kernel is getting overloaded.
This can be seen by the following results from the tcpdump from the server:
18:02:18.963370 IP xxxx > yyyy.zzz: . ack 10017011 win 0
Which data structure/strategy would you use to cache the read data to avoid the overload on C#?
I am currently working with a delegate for every socket in a seperate thread which is called after every read record, but this seems to be a problem, so that I might need to split it into two threads.
Or should I look at tweaking the TCP parameters on the Windows side? I am using Windows 7.
Are there any books for the kernel optimization?
I was really blindsided that the program got slower even though the load is around 20-25 % for the full machine and 50-60 % on the highest loaded core.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
如果不太了解你的程序的架构，我会猜测你的程序本身就是瓶颈。如果您的应用程序不使用后台工作程序来收集 TCP 消息并存储它们以供执行“繁重工作”的代码使用，那么您的程序将读取现有缓冲区，然后在 TCP 缓冲区重新填充时处理它所拥有的内容。这种恶性循环只能有一个结局：最终，在等待程序完成处理上次获得的内容时，缓冲区将溢出。
我会考虑将 TCP 接收重构为 ETL 类型的模型。您需要一个线程，其工作是侦听 TCP 输入、从缓冲区中提取数据并将其排入对程序其余部分可见的集合中。除了从缓冲区中取出数据并将原始消息推入该队列之外，它不应该执行任何其他操作，并且应该尽可能快地执行此操作。
将接收到的消息存储在内存中后，程序的其余部分几乎可以花费与消耗每条消息一样长的时间。您仍然必须确保不会超载此内存队列，但由于单个 .NET 对象最多可以引用 2GB 的内存，因此您遇到麻烦的可能性要小得多。如果程序内队列溢出，则需要考虑提高处理效率，或者在进程的第二阶段抛出更多线程。
Without knowing too much about the architecture of your program, I would guess that your program itself is the bottleneck. If your application doesn't use a background worker to collect TCP messages and store them to be consumed by the code that does the "heavy lifting", then your program is reading the existing buffer, then processing what it has while the TCP buffer refills. This vicious cycle can only have one ending; eventually the buffer will overflow while waiting for your program to finish processing what it got last time.
I would look at refactoring your TCP reception into an ETL-type model. You need one thread whose job is to listen for TCP input, extract the data from the buffer and enqueue it in a collection that's visible to the rest of the program. It should do nothing else other than yank data out of the buffer and shove the raw messages into this queue, and it should do this as fast as it can.
With the received messages in memory, the rest of your program can take almost as long as it wants to consume each message. You still have to ensure you don't overload this in-memory queue, but as a single .NET object can reference up to 2GB worth of memory, you're much less likely to run into trouble. If you ARE overflowing this in-program queue, you need to look at either making the processing more efficient, or throwing more threads at this second stage of the process.