如何将大文件从本地传输到远程盒子并自动恢复并仅传输已更改的内容?

发布于 2024-10-08 17:35:00 字数 429 浏览 1 评论 0原文

我尝试以下命令 rsync -av --progress --inplace --rsh='ssh' /home/tom/workspace/myapp.war [email protected]:/home/rtom/uploads

但当我在重新生成 myapp.war 的应用程序中进行小更改时,每次执行命令时,它似乎都会再次传输整个文件。

我还希望在连接丢失时自动恢复连接。我认为这部分正在发挥作用。

传输应通过 ssh 进行。

连接速度非常慢并且也可能中断,因此仅传输已更改的内容非常重要。当然还必须确保文件被正确传输。

I try the following command
rsync -av --progress --inplace --rsh='ssh' /home/tom/workspace/myapp.war [email protected]:/home/rtom/uploads

But it seems it transfers the whole file again each time I execute the command when I make a small change in app that regenerates the myapp.war.

I want also the connection to automatically resume if connection is lost. I think this part is working.

The transfer should occur over ssh.

The connection speed is very slow and can break too so it is important that it transfers only what has changed. Of course it must also ensure that the file was correctly transfered.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

零時差 2024-10-15 17:35:00

rsync 确实可以有效地处理文件中相对较小的更改和部分上传。 rsync 算法朝着这个方向做出了巨大的努力。

问题在于 WAR 文件是“扩展”JAR 文件,本质上是 ZIP 档案,因此被压缩。

未压缩文件中的一个小更改将更改该文件所属的整个压缩段,而且最重要的是,它还会显着更改其大小。这可以克服 rsync 检测和处理最终压缩文件中的更改的能力。

在 ZIP 存档中,每个未压缩的文件都有自己的压缩段。因此,文件在存档中的放置顺序对于实现与先前版本的一定程度的相似性也很重要。根据 WAR 文件的创建方式,仅添加新文件或重命名文件就可能导致段移动,从而导致 WAR 文件无法识别。换句话说:

应用程序中的微小更改通常意味着 WAR 文件中的相当大的更改。

rsync 并非旨在处理压缩文件中的更改。但是,它可以处理应用程序中的更改。一种解决方案是使用它上传应用程序文件,然后在远程主机上创建 WAR 文件。

一种稍微不同的方法(不需要在远程主机上使用任何开发工具)是在本地解压(即解压)WAR 文件,上传其内容,然后在远程主机上再次打包(即压缩)。此解决方案仅需要远程主机上的 zipjar 实现。

rsync does handle relatively small changes and partial uploads in a file efficiently. There has been significant effort in the rsync algorithm towards this direction.

The problem is that WAR files are "extended" JAR files, which are essentially ZIP arhives and therefore compressed.

A small change in an uncompressed file will change the whole compressed segment where that file belongs and - most importantly - it can also change its size significantly. That can overcome the ability of rsync to detect and handle changes in the final compressed file.

On ZIP archives each uncompressed file has its own compressed segment. Therefore the order in which files are placed in the archive is also important with regard to achieving a degree of similarity to a previous version. Depending on how the WAR file is created, just adding a new file or renaming one can cause segments to move, essentially making the WAR file unrecognisable. In other words:

A small change in your application normally means a rather large change in your WAR file.

rsync is not designed to handle changes in compressed files. However, it can handle changes in your application. One solution would be to use it to upload your application files and then create the WAR file on the remote host.

A slightly different approach - that does not need any development tools on the remote host - would be to unpack (i.e. unzip) the WAR file locally, upload its contents and then pack (i.e. zip) it again on the remote host. This solution only requires a zip or jar implementation on the remote host.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文