DotNetZip 中的大型文件存档的压缩问题

发布于 2024-10-14 04:56:18 字数 1257 浏览 3 评论 0原文

您好...

我正在用 c# 3.5 编写一个备份程序,使用最新的 DotNetZip。该程序的基础知识是给定服务器上的位置以及跨区 zip 文件的最大大小,然后执行。从那里它应该遍历给定位置的所有文件夹/文件并将它们添加到存档中,保持确切的结构。它还应该将所有内容压缩到合理的数量。给定的未压缩文件夹/文件集合很容易达到 10-25GB,而创建的跨文件每个限制为 1GB 左右。

我一切正常(使用 DotNetZip)。我唯一的挑战是实际上几乎没有发生任何压缩。我选择使用“AddDirectory”方法是为了简化代码,并且一般来说它看起来很适合我的项目。阅读完周围的内容后,我再次猜测这个决定。

  1. 鉴于以下代码和存档中的大量文件,我是否应该在将每个文件添加到 zip 时对其进行压缩?或者 Adddirectory 方法应该提供大约相同的压缩吗?

  2. 我已经尝试了 Ionic.Zlib.CompressionLevel 提供的每个压缩级别,但似乎都没有帮助。我是否应该考虑使用外部压缩算法并将其流式传输到我的 DotNetZip 文件中?

using (ZipFile zip = new ZipFile())  
{  
  zip.AddDirectory(root.FullName);  

  if (zipPassword.Length > 0)  
    zip.Password = zipPassword;  

  float size = zipGbSize * 1024 * 1024 * 1024;  

  zip.CompressionLevel = Ionic.Zlib.CompressionLevel.BestCompression;  
  zip.AddProgress += new EventHandler<AddProgressEventArgs>(Zip_AddProgress);  
  zip.ZipError += new EventHandler<ZipErrorEventArgs>(Zip_ZipError);  
  zip.Comment = "This zip was created at " + System.DateTime.Now.ToString("G");  
  zip.MaxOutputSegmentSize = (int)size;   //in gig  
  zip.Name = archiveDir.FullName + @"\Task_" + taskId.ToString() + ".zip";  
  zip.Save();  
}  

感谢您的帮助!

Greetings....

I am writing a backup program in c# 3.5, using hte latest DotNetZip. The basics of the program is to be given a location on a server and the max size of a spanned zip file and go. From there it should traverse all the folder/files from the given location and add them to the archive, keeping the exact structure. It should also compress everything down to a reasonable amount. A given uncompressed collection of folders/files could easily be 10-25gb, with the created spanned files being limited to about 1gb each.

I have everything working (using DotNetZip). My only challenge is there is little to no compession actually happening. I chose to use the "AddDirectory" method for simplicity of code and just generally how well it seemed to fit my project. After reading around I am second guessing that decision.

  1. Given the below code and the large amount of files in an archive, should I compress each file as it is added to the zip? or should the Adddirectory method provide about the same compression?

  2. I have tried every level of compression offered by Ionic.Zlib.CompressionLevel and none seem to help. Should I think about using an outside compression algorithm and stream it into my DotNetZip file?

using (ZipFile zip = new ZipFile())  
{  
  zip.AddDirectory(root.FullName);  

  if (zipPassword.Length > 0)  
    zip.Password = zipPassword;  

  float size = zipGbSize * 1024 * 1024 * 1024;  

  zip.CompressionLevel = Ionic.Zlib.CompressionLevel.BestCompression;  
  zip.AddProgress += new EventHandler<AddProgressEventArgs>(Zip_AddProgress);  
  zip.ZipError += new EventHandler<ZipErrorEventArgs>(Zip_ZipError);  
  zip.Comment = "This zip was created at " + System.DateTime.Now.ToString("G");  
  zip.MaxOutputSegmentSize = (int)size;   //in gig  
  zip.Name = archiveDir.FullName + @"\Task_" + taskId.ToString() + ".zip";  
  zip.Save();  
}  

Thank you for any help!

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

绮筵 2024-10-21 04:56:19

1.鉴于以下代码和存档中的大量文件,我是否应该在将每个文件添加到 zip 时对其进行压缩?

DotNetZip 的工作方式是在将每个文件添加到存档时对其进行压缩。您的应用程序不需要进行压缩。 DotNetZip 会为您完成此操作。

或者 Adddirectory 方法应该提供大约相同的压缩吗?

写入 zip 存档时,通过 AddDirectory() 方法添加到 zip 文件的条目与通过 AddFile() 添加的条目经过相同的代码路径。文件数据被压缩,然后可选地加密,然后写入 zip 文件。


主动提示:您不需要做:

zip.AddProgress += new EventHandler<AddProgressEventArgs>(Zip_AddProgress);   

您可以做:

zip.AddProgress += Zip_AddProgress;   

您如何确定没有发生压缩?

如果您对每个条目的压缩感到好奇,可以注册 SaveProgress 事件处理程序。 SaveProgress 事件在写入存档期间的不同时间触发,包括保存开始时、DotNetZip 开始写入一个条目的数据时、写入一个条目期间的不同时间间隔、完成写入每个条目的数据后,以及写入所有数据后。这些阶段在ZipProgressEventType 枚举。当EventType 为Saving_AfterWriteEntry 时,您可以计算该特定条目的压缩率。

为了验证压缩没有发生,我建议您注册这样一个 SaveProgress 事件并查看该压缩率。

此外,如上所述,某些文件类型无法压缩。 JPG、MPG、MP3、ZIP 文件和其他文件的压缩性不是很好。


最后,如果您只使用 DotNetZip 命令行工具,备份可能会容易得多。如果您只想备份特定目录,则可以使用命令行工具(zipit.exe)并避免编写程序。对于 zipit.exe 工具,如果您使用 -v 选项,该工具将打印进度报告,并将通过我上面描述的机制显示每个条目的压缩情况。即使您更喜欢编写自己的程序,您也可以考虑使用 zipit.exe 来验证使用 DotNetZip 时是否发生压缩。

1.Given the below code and the large amount of files in an archive, should I compress each file as it is added to the zip?

The way DotNetZip works is to compress each file as it is added to the archive. Your app does not need to do compression. DotNetZip does this for you.

or should the Adddirectory method provide about the same compression?

Entries added to a zip file via the AddDirectory() method go through the same code path when the zip archive is written, as entries added via AddFile(). The file data is compressed, then optionally encrypted, then written to the zip file.


an unsolicited tip: you don't need to do:

zip.AddProgress += new EventHandler<AddProgressEventArgs>(Zip_AddProgress);   

you can just do:

zip.AddProgress += Zip_AddProgress;   

how are you determining that no compression is occurring?

If you are curious about the compression on each entry, you can register a SaveProgress event handler. The SaveProgress event is fired at various times during the writing of an archive, including when saving begins, when DotNetZip begins writing the data for one entry, at various intervals during the writing of one entry, after finishing writing the data for each entry, and after finishing writing all data. These stages and described in the ZipProgressEventType enumeration. When the EventType is Saving_AfterWriteEntry, you can calculate the compression ratio for THAT particular entry.

To verify that compression is not occurring, I'd suggest that you register such a SaveProgress event and look at that compression ratio.

Also, as described above, some file types cannot be compressed. JPG, MPG, MP3, ZIP files, and others are not very compressible.


Finally, doing a backup may be lots easier to do if you just use the DotNetZip command-line tool. If all you want to do is backup a particular directory, you could use the command line tool (zipit.exe) and avoid writing a program. With the zipit.exe tool, if you use the -v option, the tool prints progress reports, and will display the compression for each entry, via the mechanism I described above. Even if you prefer to write your own program, you might consider using zipit.exe to verify that compression is, or is not, occuring when you use DotNetZip.

太阳男子 2024-10-21 04:56:19

我不确定是否低估了您的问题,但任何 zip 文件的最大大小 其 4Gb 。也许每次达到该限制时,您都必须创建一个新的 ZipFile

抱歉,如果这对您没有帮助。

Im not sure to have understated your question, but the maximum size for any zip file its 4Gb. Maybe you have to create a new ZipFile every time you reach that limit.

Sorry if that doesnt help you.

吻风 2024-10-21 04:56:19

您要压缩什么类型的数据?某些类型的数据压缩效果不佳,例如 JPEG 或已压缩的 ZIP 文件。

What sort of data are you compressing? Some sorts of data just doesn't compress very well, for example JPEGs, or ZIP files which are already compressed.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文