使用 BitTorrent 在多个站点之间进行文件复制/同步

发布于 2024-10-13 12:36:47 字数 142 浏览 7 评论 0原文

我需要构建一个依赖于站点之间大文件复制的分布式系统。

我想到使用像 BitTorrent 这样的 p2p 技术来节省带宽并提高可靠性。

我是不是大错特错了?

有人设计过这样的解决方案吗?

你推荐哪些图书馆?

I need to build a distributed system which relies on replication of large files between the sites.

I thought of using p2p technology like bittorrent to save on bandwidth, and increase reliability.

Am I terribly wrong?

Have anyone ever architected such a solution?

What libraries do you recommend?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

流心雨 2024-10-20 12:36:47

BitTorrent 开发人员提出了一个新的有前景的解决方案:BitTorrent Sync

它具有以下特点:

  • 无限且免费!
  • 目前支持 Windows、Mac 和 Linux。移动平台正在开发中。
  • 专为处理大文件而设计。
  • 私密且安全:所有流量均经过加密。
  • 对等发现协议。
  • 支持断开节点的流量中继。

A new promising solution from the developers of BitTorrent: BitTorrent Sync.

It has the following features:

  • Unlimited and free!
  • Currently supports Windows, Mac and Linux. Mobile platforms are in the works.
  • Specifically designed to handle large files.
  • Private and secure: all traffic is encrypted.
  • Peer discovery protocols.
  • Supports traffic relay for disconnected nodes.
白首有我共你 2024-10-20 12:36:47

我刚刚从 Twitter 上找到了这个开源项目,它非常完美:

https://github.com/lg/murder

来自文档:

Murder是一种使用 Bittorrent 将文件分发到生产环境中的大量服务器的方法。这允许在数百到数万台服务器的环境中进行可扩展和快速的部署,否则集中式分发系统将无法正常工作。 “谋杀”通常用来指一群乌鸦,在这种情况下适用于一群服务器在做某事。

I just found this open-source project from Twitter which hits the nail perfectly:

https://github.com/lg/murder

From the docs:

Murder is a method of using Bittorrent to distribute files to a large amount of servers within a production environment. This allows for scaleable and fast deploys in environments of hundreds to tens of thousands of servers where centralized distribution systems wouldn't otherwise function. A "Murder" is normally used to refer to a flock of crows, which in this case applies to a bunch of servers doing something.

尝蛊 2024-10-20 12:36:47

如果您有超过 2 个站点,那么 p2p 是更好的解决方案恕我直言。

只需安装rtorrent

为了保护第三方 torrent 客户端的内容,请在生成 .torrent 文件时设置私有标志并使用您自己的跟踪器。 opentracker是一个不错的选择。

还有一个提示:如果您的 torrent 客户端支持超级播种模式(又名BEP-16 或初始播种),启用它。它将有助于在节点之间以最少的重复分发内容。

If you have more than 2 sites, then p2p is better solution IMHO.

Just install rtorrent, deluge or any other high-performance torrent client at every site. Than you can distribute only .torrent files with scp/sftp and enjoy.

In order to secure the content from third party torrent clients, set the private flag when producing the .torrent file and use your own tracker. opentracker is a good choice.

One more hint: if your torrent client supports super seeding mode (aka BEP-16 or initial seeding), enable it. It'll help distribute the content with minimum duplication between nodes.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文