ReadFile FILE_FLAG_NO_BUFFERING 如何读取两个扇区之间的数据
我正在使用 Windows ReadFile 函数使用 FILE_FLAG_NO_BUFFERING 顺序读取 4GB 文件。我使用的是 64K 的缓冲区，一切正常，但问题是我的数据在当前缓冲区的末尾和下一次读取之间被切断。例如，我有一个 4 字节浮点数序列，当我到达最后一个浮点数时，当前缓冲区中只有三个字节，下一个字节将进入下一个读取缓冲区。那么我该如何处理这个问题呢？我是否应该跟踪保留最后三个字节所消耗的字节数，然后在读取下一个缓冲区后附加最后一个字节？或者也许将缓冲区复制到另一个缓冲区并在那里跟踪浮动？但这不就打败了不使用Windows缓存读取的优势吗？感谢您的任何帮助。
I´m using Windows ReadFile function to read a 4GB files sequentially, with FILE_FLAG_NO_BUFFERING. I´m using a buffer of 64K, and all works right, but the problem is that my data is cut between the end of the current buffer and the next read. For example, I have a sequence of 4 bytes float numbers, and when I arrive to the last float only three bytes are in the current buffer, and the next byte will come in the next read buffer. So how can I handle this? Should I track the number of bytes consumed to keep my last three bytes and then append the last one after read the next buffer? Or maybe copy the buffer to another and make the tracking of the floats there? But this doesn´t defeat the advantage of not reading with Windows cache? Thanks for any help.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
我认为，在边界上的特殊情况下，您应该将前一个块中的位和下一个块中的位并排复制到一个小的暂存空间中，然后从那里将它们整体读取。当您不在边界上时，无需使用此暂存空间。
I think that in the special case where you are on the boundary, you should copy the bits from the previous block and the bits from the next block side-by-side into a small scratch space, and read them from there in one piece. When you are not on the boundary, you need not use this scratch space.
无论如何，您必须自己进行此核算，因为需要对齐未缓冲的读取。
但更好的问题是，您认为为什么需要这样做？您是否尝试过使用 std::ifstream 来读取文件？现代处理器和缓存在隐藏（或者实际上消除）您可能认为缓冲 I/O 执行的额外副本方面大有帮助。
另外，如果您按顺序读取，
FILE_FLAG_NO_BUFFERING将抑制操作系统的预读机制。几乎可以肯定，这将使您付出的代价远远超过绕过操作系统缓冲区所获得的收益。我怀疑您会发现最简单的代码对于顺序读取大文件效果最好。这几乎就是现代系统在各个级别上进行优化的情况......
One way or another, you have to do this accounting yourself, since unbuffered reads are required to be aligned.
But a better question is, why do you think you need to do this? Have you tried using
std::ifstreamto read your file? Modern processors and caches go a long way toward hiding (or, in effect, eliminating) the extra copy that you probably think buffered I/O performs.Also, if you are reading sequentially,
FILE_FLAG_NO_BUFFERINGwill inhibit the OS's read-ahead machinery. This will almost certainly cost you far more than you will gain from bypassing the OS buffers.I suspect you will find that the simplest code will perform best for sequential reading of large files. That is pretty much the case modern systems are optimized for at every level...