没有字典的压缩
我一直在测试使用木准文件的各种压缩算法，并在ZSTD上解决。
据我了解，据我了解，除非明确指定一个ZSTD，否则使用自适应词典，因此它以空的字典开头。但是，当具有字典时，启用了压缩大小，并且执行时间非常令人满意。
与使用自适应的文件大小相比，不使用字典的文件大小要小得多。 （名称末尾的数字是压缩级别）：
- 名称：c：\ parquetfiles \ zstd1执行时间：279 ms大小：13738134
- 名称：c：\ parquetfiles \ ZSTD2执行时间：140 ms size：13207017名称：C 13207017名称
- ：C 13207017名称：C :\ParquetFiles\Zstd9 Execution time: 511 ms Size: 12701030
And for comparison the log from using the adaptive dictionary:
- Name: C:\ParquetFiles\ZstdDictZstd1 Execution time: 487 ms Size: 19462825
- Name: C:\ParquetFiles\ZstdDictZstd2 Execution time ：402 ms尺寸：19292513
- 名称：c：\ parquetfiles \ zstddictzstd9执行时间：614 ms大小：19072779
您能帮助我理解此意义的意义，不应使用空词典的输出，至少像ZSTD压缩一样好，词典禁用？
I have been testing the various compression algorithms with parquet files, and have settled on Zstd.
Now as far as I understand Zstd uses adaptive dictionary unless one is explicitly specified, thus it begins with an empty one. However when having a dictionary enabled the compressed size and and the execution time are quite unsatisfactory.
The file size without using a dictionary is quite less compared to using the adaptive one. (The number at the end of the name is the compression level):
- Name: C:\ParquetFiles\Zstd1 Execution time: 279 ms Size: 13738134
- Name: C:\ParquetFiles\Zstd2 Execution time: 140 ms Size: 13207017
- Name: C:\ParquetFiles\Zstd9 Execution time: 511 ms Size: 12701030
And for comparison the log from using the adaptive dictionary:
- Name: C:\ParquetFiles\ZstdDictZstd1 Execution time: 487 ms Size: 19462825
- Name: C:\ParquetFiles\ZstdDictZstd2 Execution time: 402 ms Size: 19292513
- Name: C:\ParquetFiles\ZstdDictZstd9 Execution time: 614 ms Size: 19072779
Can you help me understand the significance of this, shouldn't the output with an empty dictionary perform at least as good as Zstd compression with dictionary disabled?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论