加载压缩文件流的特定部分
我们有一个简单的二进制文件格式,用于在我们的应用程序(C# .NET Windows 应用程序)中缓存数据。该格式基本上是一个简短的内容,表示对象类型,后跟对象 id 的 guid(字符串),然后是任何对象特定数据(字符串、整数等)。我们希望能够在同一个文件中存储许多对象(> 10000),但在某些情况下仅按需加载。我们的解决方案是保留文件中对象位置的索引 - 因此,当我们开始写入新对象时,我们记录对象在文件流中开始的位置。当我们想要加载这个对象时,我们使用这个索引位置来加载相关数据。这很好用。
但是,如果我们想压缩文件,这种方法还可以吗?我不太热衷于压缩的工作原理,特别是我们计划使用的 GZipStream 类 (System.IO.Compression)。据我了解,此类不支持 Seeking 或 Position 属性。是否仍然可以使用底层 FileStream 的 Seek 和 Position(我猜不能)?基本上,是否有可能有一个我们可以有选择地加载的压缩文件,如果可以,我们该怎么做?
谢谢,
史蒂夫
We have a simple binary file format for caching data in our application (C# .NET Windows App). The format is basically a short that indicates the object type followed by a guid (string) for the object id then any object specific data (strings ints whatever). We want to be able to store many objects in the same file (> 10000) but in certain situations only load on demand. The solution we have is to keep an index of the object locations within the file - so when we start writing a new object we record the position in the file stream the object starts. When we want to load this object, we use this indexed location to load the relavent data. This works fine.
However, if we want to compress the file, will this method still be possible? I'm not too hot on how compression works and specifically the GZipStream class (System.IO.Compression) that we are planning to use. As I understand it, this class does not support Seeking or the Position property. Will it still be possible to use the Seek and Position of the underlying FileStream (I'm guessing not)? Basically, is it possible to have a compressed file that we can selectively load from, if so, how do we do it?
Thanks,
Steve
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
不,如果您想访问未压缩数据中的特定位置,您必须解压缩它,至少暂时解压缩
No, if you want to access a specific position in the uncompressed data you will have to decompress it, at least temporarily
这不是真正的 Seek,但解决方案是:
显然这根本不是一个理想的解决方案,但它仍然可以提供可接受的性能。
就我而言,压缩文件很容易适合内存(未压缩的则不然),因此我已将压缩文件加载到内存中。
理想情况下,底层的 deflate 类将被更改以支持真正的 Seeks。
This is not a true Seek, but a solution would be:
Obviously this is not at all an ideal solution, but it may still give acceptable performance.
In my case, the compressed file fits easily within memory (uncompressed does not), so I have loaded the compressed file into memory.
Ideally the underlying deflate class would be changed to support true Seeks.
更好的解决方案:
使用 GZipStream 在内存中创建压缩字节,然后编写自己的类来控制缓存并将其写入磁盘(不要使用 DeflateStream)。还可以编写您自己的类以从磁盘读取此数据。
然后,您可以确保底层磁盘流支持搜索。
a better solution:
use GZipStream to create compressed bytes within memory, and then write your own class to control the caching and writing of this to disk (don't use DeflateStream). Also write your own class to for the reading of this data from disk.
You can then ensure the underlying disk stream supports Seeks.