调整缓冲区长度以从 NetworkStream 读取小数据

发布于 2024-12-18 04:24:27 字数 1118 浏览 1 评论 0原文

从 TcpClient/NetworkStrem 读取小数据时如何微调 bufferSize? 如果 bufferSize 很大,例如 1024、4096,则 Read/BeginRead 会阻塞。 如果我将 bufferSize 设置为 16、32,它就不会阻塞。

  • 将 bufferSize 设置为 1 是否可以保证不会出现任何块?性能影响会不会很差?
  • 看起来将 ReadTimeout 设置为 1000、2000 之类的值有 没有超过阻塞的效果。有没有其他方法可以阻止 矮吗? (NoDelay = true 不起作用)

    public static IObservable; AsyncReadChunk(此 Stream 流,int bufferSize)
    {
        var buffer = new byte[bufferSize];
    
        返回 Observable.FromAsyncPattern(stream.BeginRead, stream.EndRead)(buffer, 0, bufferSize)
            .Select(cbRead =>
                        {
                            var dataChunk = 新字节[cbRead];
    
                            Buffer.BlockCopy(缓冲区, 0, dataChunk, 0, cbRead);
    
                            返回数据块;
                        });
    }
    
    公共静态 IObservable AsyncRead(此 NetworkStream 流,int bufferSize)
    {
        return Observable.Defer(() => stream.DataAvailable ? AsyncReadChunk(stream, bufferSize) : Observable.Return(new byte[0]))
            。重复()
            .TakeWhile((dataChunk, index) => dataChunk.Length > 0);
    }
    

How to fine tune the bufferSize while reading small data from the TcpClient/NetworkStrem?
If the bufferSize is big like 1024, 4096 the Read/BeginRead blocks.
If I set the bufferSize to 16, 32 it works without blocking.

  • Does setting the bufferSize to 1 guarantee there won't be any blocks? Will the performance impact be very bad?
  • It looks like setting the ReadTimeout to values like 1000, 2000 has
    no effect over blocking. Is there any other way to make the blocking
    be short? (NoDelay = true doesn't work)

    public static IObservable<byte[]> AsyncReadChunk(this Stream stream, int bufferSize)
    {
        var buffer = new byte[bufferSize];
    
        return Observable.FromAsyncPattern<byte[], int, int, int>(stream.BeginRead, stream.EndRead)(buffer, 0, bufferSize)
            .Select(cbRead =>
                        {
                            var dataChunk = new byte[cbRead];
    
                            Buffer.BlockCopy(buffer, 0, dataChunk, 0, cbRead);
    
                            return dataChunk;
                        });
    }
    
    public static IObservable<byte[]> AsyncRead(this NetworkStream stream, int bufferSize)
    {
        return Observable.Defer(() => stream.DataAvailable ? AsyncReadChunk(stream, bufferSize) : Observable.Return(new byte[0]))
            .Repeat()
            .TakeWhile((dataChunk, index) => dataChunk.Length > 0);
    }
    

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

凉栀 2024-12-25 04:24:28

看起来像是设置
ReadTimeout 设置为 1000、2000 等值对阻塞没有影响。

Msdn 说,使用异步 BeginRead 方法时,ReadTimeout 不起作用。

将 bufferSize 设置为 1 是否可以保证不会出现任何块?

不,当然不是。当没有通过连接发送单个字节时,无论缓冲区大小如何,Read 调用都会阻塞。

性能影响会很严重吗?

我猜你在这里谈论的是 1 字节缓冲区。
这取决于您接收的数据量和频率以及您在 EndRead 上执行的代码。当处理高带宽的流时,影响可能很大。你必须在接收数据时尝试观察你的CPU。

我不太确定您想要实现什么目标以及您对阻止的担忧是什么。

当您使用 1024 字节的缓冲区启动 Receive(或 Networkstream.Read)时,并且您在套接字上接收 10 字节,Read 调用将在短暂延迟后返回该 10 字节,但在整个缓冲区填满之前不会阻塞。

还有其他方法可以缩短阻塞时间吗?

是什么意思。正如我所说,即使有很大的缓冲区,读取在接收少量数据时也不会无限阻塞。

(NoDelay = true 不起作用)

完全不同的故事,但在发件人端将其设置为 true 可能会很有趣(如果您也可以控制的话)。

当设置为 false(默认值)时,它将把发送的小数据块组合成大数据块,以减少一个 tcp 数据包(40 字节标头)的开销。

编辑

我的意思是 NetworkStream.BeginRead 在没有数据的情况下立即返回。

使用stream.DataAvailable怎么样?当没有数据时它应该返回 false。

此外,当使用异步模式时,调用会阻塞直到有事情要做,这不是预期的行为吗?否则,您将在繁忙的循环中进行主动轮询。

当缓冲区很大时,有时会等待 60 秒才能返回(当缓冲区未完全填充或填充到一定数量时)

嗯,简直不敢相信。您通过通道发送什么类型的数据?每分钟 1 个字节?每秒 1000 字节?

It looks like setting the
ReadTimeout to values like 1000, 2000 has no effect over blocking.

Msdn says, that ReadTimeout has no effect when using the async BeginRead method.

Does setting the bufferSize to 1 guarantee there won't be any blocks?

No, of cause not. When not a single byte is sent over the connection, the Read call will block regardless of the buffersize.

Will the performance impact be very bad?

I guess you are talking about the 1 byte buffer here.
It depends on the amount and frequency of data you receive and what code you execute on EndRead. When processing a stream with a high bandwidth, the impact can be big. You have to try and watch your cpu while receiving data.

I'm not really sure what you want to achieve respectively what your concern about blocking is.

When you start a Receive (or Networkstream.Read) with a buffer of say 1024 bytes, and you receive 10 Bytes on the socket, the Read call will return that 10 Bytes after a short delay but will not block until the whole buffer is filled.

Is there any other way to make the blocking be short?

What do you mean with short. As i say, even with a big buffer, the Read won't block endlessly when receiving small amounts of data.

(NoDelay = true doesn't work)

That a completely different story but is might be interesting to set it to true on your senders side (if you are in control of that, too).

When set to false (the default), it will combine small datachunks being sent into bigger ones to reduce the overhead of one tcp packet (40 Bytes header).

EDIT

I mean NetworkStream.BeginRead to return immediately if there is no data.

What about using stream.DataAvailable? It should return false when there is no data.

In addition, when using the async pattern, isn't it expected behaviour that the calls will block until there is something to do? Otherwise you will get active polling in a busy loop.

When the buffer is big it sometimes waits 60 seconds to return (when it is not fully filled or filled to some amount)

Hm, can't believe that. What kind of data are you sending over the channel? 1 byte every minute? 1000 bytes per second?

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文