带流的文件 I/O - 最佳内存缓冲区大小
我正在编写一个小型 I/O 库来协助完成一个更大的（业余爱好）项目。该库的一部分对文件执行各种功能，该文件通过 FileStream 对象读取/写入。在每个 StreamReader.Read(...) 传递中，
我都会触发一个事件，该事件将在主应用程序中用于显示进度信息。循环中进行的处理是可行的，但并不太耗时（例如，它可能只是一个简单的文件副本，或者可能涉及加密......）。
我的主要问题是：使用的最佳内存缓冲区大小是多少？考虑到物理磁盘布局，我可以选择 2k，它可以覆盖 CD 扇区大小，并且是 512 字节硬盘扇区的一个很好的倍数。在抽象树的更高层，您可以选择更大的缓冲区，它可以一次读取整个 FAT 簇。我意识到对于今天的 PC，我可以选择更耗内存的选项（例如几个 MiB），但随后我增加了 UI 更新之间的时间，并且用户会感觉到应用程序响应速度较慢。
顺便说一句，我最终希望为 FTP/HTTP 服务器（通过本地网络/快速 DSL）托管的文件提供类似的界面。对于这些来说，最佳的内存缓冲区大小是多少（同样，感知响应能力与性能之间的“最佳情况”权衡）？
I am writing a small I/O library to assist with a larger (hobby) project. A part of this library performs various functions on a file, which is read / written via the FileStream object. On each StreamReader.Read(...) pass,
I fire off an event which will be used in the main app to display progress information. The processing that goes on in the loop is vaired, but is not too time consuming (it could just be a simple file copy, for example, or may involve encryption...).
My main question is: What is the best memory buffer size to use? Thinking about physical disk layouts, I could pick 2k, which would cover a CD sector size and is a nice multiple of a 512 bytes hard disk sector. Higher up the abstraction tree, you could go for a larger buffer which could read an entire FAT cluster at a time. I realise with today's PC's, I could go for a more memory hungry option (a couple of MiB, for example), but then I increase the time between UI updates and the user perceives a less responsive application.
As an aside, I'm eventually hoping to provide a similar interface to files hosted on FTP / HTTP servers (over a local network / fastish DSL). What would be the best memory buffer size for those (again, a "best-case" tradeoff between perceived responsiveness vs. performance)?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
文件已被文件系统缓存缓冲。您只需选择一个不会强制 FileStream 频繁调用本机 Windows ReadFile() API 来填充缓冲区的缓冲区大小。不要低于 1 KB，超过 16 KB 会浪费内存，并且对 CPU 不友好 L1 缓存（通常为 16 或 32 KB 数据）。
4 KB 是传统选择，尽管这只是偶然地恰好跨越虚拟内存页面。分析起来很困难；您最终将测量读取缓存文件所需的时间。如果数据在缓存中可用，则其运行速度为 5 GB/秒或更高。第二次运行测试时它将位于缓存中，并且这种情况在生产环境中不会经常发生。文件 I/O 完全由磁盘驱动器或 NIC 控制，并且速度非常慢，复制数据是花生。 4 KB 就可以了。
Files are already buffered by the file system cache. You just need to pick a buffer size that doesn't force FileStream to make the native Windows ReadFile() API call to fill the buffer too often. Don't go below a kilobyte, more than 16 KB is a waste of memory and unfriendly to the CPU's L1 cache (typically 16 or 32 KB of data).
4 KB is a traditional choice, even though that will exactly span a virtual memory page only ever by accident. It is difficult to profile; you'll end up measuring how long it takes to read a cached file. Which runs at RAM speeds, 5 gigabytes/sec and up if the data is available in the cache. It will be in the cache the second time you run your test, and that won't happen in a production environment too often. File I/O is completely dominated by the disk drive or the NIC and is glacially slow, copying the data is peanuts. 4 KB will work fine.
当我直接通过流对象处理文件时，我通常使用 4096 字节。它似乎在多个 I/O 区域（本地文件系统、LAN/SMB 上相当有效） 、网络流等），但我还没有对其进行概要分析或任何其他内容。很久以前，我看到几个例子使用了这个尺寸，它就留在了我的记忆中。但这并不意味着它是最好的。
When I deal with files directly through a stream object, I typically use 4096 bytes. It seems to be reasonably effective across multiple I/O areas (local file system, LAN/SMB, network stream, etc.), but I haven't profiled it or anything. Way back when, I saw several examples use that size, and it stuck in my memory. That doesn't mean it's the best though.
“这取决于”。
您必须使用不同的缓冲区大小来测试您的应用程序，以确定哪个是最好的。你无法提前猜测。
"It depends".
You would have to test your application with different buffer sizes to determine whis is best. You can't guess ahead of time.
我认为默认值通常是最好的 - 因此我使用基于
内部const int变量DefaultBufferSize的4096BFileStream 类。I suppose that default value is usually the best - therefore i use
4096Bbased oninternal const intvariableDefaultBufferSizein FileStream class.