将大数据从MongoDB迁移到不同服务器上的另一个MongoDB
我有两个MongoDB簇，它们在4个服务器上运行（每个群集上的2个服务器）。它们与集合模式相同。现在，我想将新数据从一个群集迁移到另一个群集的现有旧数据。
我需要将数据从一个群集迁移到另一个群集（约700GB）。
我已经尝试使用MongoDB指南针的导入/导出DB工具，例如Mongodump/Mongorestore，Mongoexport/Mongoimport。它们都按预期进行测试。但这是小数据。有了大数据，它们将从DB源和导入到目标BD的导出速度非常慢。也许它们会在导出/导入大数据时崩溃。 那么，我能做到的最好的方法是什么？我的想法是将复制机制用于性能和安全原因。
- 将源群集的一台Mongo服务器作为主要实例（服务器A），以及一个目标群集作为辅助实例（服务器B），然后在同一replicaset上加入它们。问题是源服务器上的700GB新数据将同步到目标服务器，而无需通过副本同步机制删除现有数据？
- 如果步骤1有效，则在副本同步完成后，我将关闭服务器A和服务器B之间的replicaset，并以服务器B为主要实例在目标群集上为2个服务器重新连接副本集，而目标群集的第2服务器作为目标群次要实例。在服务器B上同步的新数据将同步到此次级实例。
那是我的想法，我不知道它会在大数据上使用。对我有什么建议吗？感谢大家的任何建议。
I have two clusters of MongoDB which run on 4 servers (2 servers on each cluster). They are the same as collections schema. And now I want to migrate new data from one cluster to another which existing old data.
I need to migrate data from one cluster to another with large data (about 700GB).
I've tried using import/export of MongoDB Compass, DB tools such as mongodump/mongorestore, mongoexport/mongoimport. All of them work as expected for testing. But it is small data. With large data, they will be very slow to export from DB source and import to destination BD. And maybe they will crash while export/import large data.
So, What is the best way I can do that? I've an idea to use replica sync mechanism for performance and safe reason.
- Use one mongo server of the source cluster as a Primary instance (server A), and one of the destination clusters as a Secondary instance (server B) and join them on the same replicaset. And the question is 700GB of new data on source server will sync to the destination server without removing existing data by replica sync mechanism or not?
- If step 1 works, after replica sync is complete, I will turn off replicaset between server A and server B and re-config the replica set for 2 servers on destination cluster with server B as Primary instance, and 2nd server of destination cluster as the Secondary instance. New data which be synced on server B will sync to this secondary instance.
That is my idea, and I don't know it will work on large data or not. Have any suggestions for me in my case? Thanks everyone for any suggestions.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
选项1 ：（小数据库大小＆lt; 500GB）
Mongodump/Mongorestore -＆gt;如果您的数据库很小，并且还有其他需要保留在目标服务器中的删除术。
选项2 ：（任何大小）
将目标服务器添加为辅助服务/隐藏在源中，它将同步一段时间，后来将其从源中拆分，然后将其与{force：true}重新配置为主要，然后您可以添加更多成员在新的集群中进行冗余。
选项3 ：（大数据库大小＆gt; 500GB）
做文件系统快照，复制到新服务器，重新配置为新服务器。
Option 1: ( small database size < 500GB )
mongodump/mongorestore -> if you have small database and you have other detabases that need to be preserved in the targeted server.
Option 2: ( any size )
Add the targeted server as SECONDARY/HIDDEN to the source , it will sync for some time , later just split it from the source and reconfigure with {force:true} as PRIMARY , then you can add more members in the new cluster for redundancy.
Option 3: ( big database size > 500GB )
Do file system snapshot , copy to the new server , reconfigure as new server.