如何将大文件从本地传输到远程盒子并自动恢复并仅传输已更改的内容?
我尝试以下命令 rsync -av --progress --inplace --rsh='ssh' /home/tom/workspace/myapp.war [email protected]:/home/rtom/uploads
但当我在重新生成 myapp.war 的应用程序中进行小更改时,每次执行命令时,它似乎都会再次传输整个文件。
我还希望在连接丢失时自动恢复连接。我认为这部分正在发挥作用。
传输应通过 ssh 进行。
连接速度非常慢并且也可能中断,因此仅传输已更改的内容非常重要。当然还必须确保文件被正确传输。
I try the following command
rsync -av --progress --inplace --rsh='ssh' /home/tom/workspace/myapp.war [email protected]:/home/rtom/uploads
But it seems it transfers the whole file again each time I execute the command when I make a small change in app that regenerates the myapp.war.
I want also the connection to automatically resume if connection is lost. I think this part is working.
The transfer should occur over ssh.
The connection speed is very slow and can break too so it is important that it transfers only what has changed. Of course it must also ensure that the file was correctly transfered.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
rsync 确实可以有效地处理文件中相对较小的更改和部分上传。 rsync 算法朝着这个方向做出了巨大的努力。
问题在于 WAR 文件是“扩展”JAR 文件,本质上是 ZIP 档案,因此被压缩。
未压缩文件中的一个小更改将更改该文件所属的整个压缩段,而且最重要的是,它还会显着更改其大小。这可以克服 rsync 检测和处理最终压缩文件中的更改的能力。
在 ZIP 存档中,每个未压缩的文件都有自己的压缩段。因此,文件在存档中的放置顺序对于实现与先前版本的一定程度的相似性也很重要。根据 WAR 文件的创建方式,仅添加新文件或重命名文件就可能导致段移动,从而导致 WAR 文件无法识别。换句话说:
应用程序中的微小更改通常意味着 WAR 文件中的相当大的更改。
rsync
并非旨在处理压缩文件中的更改。但是,它可以处理应用程序中的更改。一种解决方案是使用它上传应用程序文件,然后在远程主机上创建 WAR 文件。一种稍微不同的方法(不需要在远程主机上使用任何开发工具)是在本地解压(即解压)WAR 文件,上传其内容,然后在远程主机上再次打包(即压缩)。此解决方案仅需要远程主机上的
zip
或jar
实现。rsync
does handle relatively small changes and partial uploads in a file efficiently. There has been significant effort in the rsync algorithm towards this direction.The problem is that WAR files are "extended" JAR files, which are essentially ZIP arhives and therefore compressed.
A small change in an uncompressed file will change the whole compressed segment where that file belongs and - most importantly - it can also change its size significantly. That can overcome the ability of
rsync
to detect and handle changes in the final compressed file.On ZIP archives each uncompressed file has its own compressed segment. Therefore the order in which files are placed in the archive is also important with regard to achieving a degree of similarity to a previous version. Depending on how the WAR file is created, just adding a new file or renaming one can cause segments to move, essentially making the WAR file unrecognisable. In other words:
A small change in your application normally means a rather large change in your WAR file.
rsync
is not designed to handle changes in compressed files. However, it can handle changes in your application. One solution would be to use it to upload your application files and then create the WAR file on the remote host.A slightly different approach - that does not need any development tools on the remote host - would be to unpack (i.e. unzip) the WAR file locally, upload its contents and then pack (i.e. zip) it again on the remote host. This solution only requires a
zip
orjar
implementation on the remote host.