内存映射文件中的数据是否保证按顺序刷新?
我正在尝试实现一种文件存储机制,该机制在单个文件中保存许多可变大小的记录,并保证即使系统在硬件级别出现故障,记录集也始终可以恢复到一致的状态。
到目前为止,我提出的每个方案都以顺序写入数据为中心。一些数据将被附加到每个记录的末尾,以确认写入成功。然而,如果在刷新时数据不一定被顺序写入盘,则确认数据可以在内容数据之前被写入。
有两种明显的方法可以解决这个问题,但这两种方法都是不可取的:
- 刷新内容,然后写入确认并刷新它。添加额外的刷新可能会降低性能。
- 在确认中包含校验和(需要阅读内容以确认其有效)。
我在 Windows(32 位和 64 位)上使用 C# 和 .Net 4.0 的内存映射文件实现
I'm trying to implement a file storage mechanism which holds a number of variably-sized records in a single file with the guarantee that the set of records will always be recoverable into a consistent state, even if the system failed at a hardware level.
So far, every scheme I've come up with pivots on writing data sequentially. Some piece of data would be appended to the end of each record which confirms that the write succeeded. However, if the data is not necessarily written to the disk sequentially when flushed then it would be possible for the confirmation data to be written before the content data.
There are two obvious ways around this, but both are undesirable:
- Flush the content, then write the confirmation and flush it. Adding the extra flush may degrade performance.
- Include a checksum in the confirmation (would require reading the content to confirm that it is valid).
I'm using C# on Windows (32 and 64-bit) and .Net 4.0's memory mapped file implementation
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
这对于 C# 来说级别太低并且是特定于操作系统的。尝试使用 C 语言的 Windows API,并仔细阅读 API 规范。
This is too low level and OS specific for C#. try using Windows APIs from C, and read very carefully the API specifiations.
您是否尝试过在底层文件流上使用 FileOptions.WriteThrough ?这可能会有所帮助,因为它会禁用缓冲。其他想法是保留一个包含确认的单独文件作为最后写入的偏移量,如果它与您的文件大小不匹配(例如由于断电),您可以简单地截断它以达到该大小
Have you tried using the FileOptions.WriteThrough on the underlying filestream? That might help since it disables buffering. Other ideas would be to keep a separate file containing the confirmations as the last written offset, if it doesnt match your filesize (due to power outage for example ) you can simply truncate it do that size