文件传输协议选项?
我正在寻找一种好方法,将大量数据(10G > x > 10MB)从一台机器传输到另一台机器,可能通过多个会话。
我简要地查看了
- *ftp (sftp, tftp, ftp)
- http
- torrents (因为我一般不会有种子网络)
- rsync (不确定我是否真的能适应我的需要)
还有其他协议吗那里可能更符合要求? 上述大多数内容本身都不是非常容错的,而是依赖客户端/服务器应用程序来弥补这一不足。 在这个阶段,我更关心协议本身,而不是运行良好的特定客户端/服务器实现。
(是的,我知道我可以通过 udp 编写自己的程序,但我更喜欢几乎任何其他东西!!)
I am looking for a good way to transfer non-trivial (10G > x >10MB) amounts of data from one machine to another, potentially over multiple sessions.
I have looked briefly at
- *ftp (sftp, tftp, ftp)
- http
- torrents (out because I will not have a seed network in general)
- rsync (not sure if I can really adapt this to what I need)
Are there any other protocols out there that might fit the bill a little better? Most of the above are not very fault tolerant in and of themselves, but rather rely on client/server apps to pick up the slack. At this stage I care much more about the protocol itself, rather than a particular client/server implementation that works well.
(And yea I know I can write my own over udp, but I'd prefer almost anything else!!)
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(5)
我使用 rsync(通过 SSH)传输任何我认为可能需要一分钟以上的内容。
可以轻松地进行速率限制、暂停/恢复以及获取进度报告。 您可以使用 SSH 密钥将其自动化。 它(通常)已经安装了(无论如何,在 *nix 盒子上)。
根据您的需要,rsync 可能可以适应。 如果您要分发给很多用户,出于防火墙考虑,FTP/HTTP 可能更好; 但 rsync 非常适合一对一或一对几个传输。
I use rsync (over SSH) to transfer anything that I think might take more than a minute.
It's easy to rate-limit, suspend/resume and get progress reports. You can automate it with SSH keys. It's (usually) already installed (on *nix boxes, anyway).
Depending on what you need, rsync can probably adapt. If you're distributing to a lot of users, FTP/HTTP might be better for firewall concerns; but rsync is great for one-to-one or one-to-a-few transfers.
rsync 几乎总是最好的选择。
由于它仅传输差异,因此如果传输中断,下次传输将不会与第一次有太大不同(当目的地没有文件时)
rsync is almost always the best bet.
since it transfers only differences, if the transfer is interrupted, the next time it won't be so different as the first one (when there wasn't a file at destination)
BitTorrent 不需要大型种子网络即可发挥作用 - 只需一名播种者和一名对等点即可正常工作。 设置跟踪器等会产生一点开销,但一旦设置完毕,这将是一种很好的、快速的、容错的传输方法。
BitTorrent doesn't require a big seed network to be effective - it'll work just fine with one seeder and one peer. There's a little bit of overhead setting up a tracker etc., but once set up it'd be a nice, zippy, fault-tolerant method of transferring.
嗯,HTTP 是一个不错的选择,因为它支持使用字节范围重新启动部分传输。 FTP 或 TFTP 很好,因为您可以获得配置极其简单的服务器软件,而不必锁定 HTTP 服务器之类的东西。
Well, HTTP is a good option, in that it supports restarting partial transfers by using byte ranges. FTP or TFTP are good because you can get server software that's extremely simple to configure, rather than having to lock down something like an HTTP server.
Argonne 正在使用 GridFTP 来可靠地传输大量数据。
GridFTP is what Argonne is using to transport huge amounts of data reliably.