如何重新启动应用程序而不丢失内存中保存的 TreeMap?
在 Spring Boot 应用程序中,我在内存中保存了一个 TreeMap。我每秒执行大约 10,000 次操作,并且可能会增加。为了提高性能,我将数据保存在内存中。我希望我的应用程序能够在应用程序重新启动时从相同的状态启动。
我可以为此找到一些方法。
在 Hazelcast 上保存数据。 在这种情况下,除非 Hazelcast 死亡,否则我不会冒丢失数据的风险,但如果 Hazelcast 死亡,我将无法恢复数据。此外,我认为在 Hazlecast 上同步这么多操作没有意义。
正在将事件同步到数据库。 在这里,我的数据丢失的风险非常低。但是,我需要在每次操作后执行查询。这可能会影响性能。另外,我需要处理数据库更新的异常。
批量同步数据 我在这里只能找到一个现成的解决方案,MapDB。我正打算尝试一下,但我还没有尝试过。如果有一个更可靠、优化的接收器解决方案,也使用 db 而不是 file,我更愿意使用它。
有什么建议可以解决这个问题吗?
In a Spring Boot application, I keep a TreeMap in memory. I'm doing around 10,000 operations per second, and it may increase. To improve performance, I kept data in memory. I want my app to be able to start from the same state when application is restarted.
There are some methods I could find for this.
Keeping data on Hazelcast.
In this case I don't risk losing the data unless the Hazelcast dies, but if the Hazelcast dies, I can't restore data. Additionally, I don't think it makes sense to sync that amount of operations on Hazlecast.Synchronizing events to database.
Here, my risk of data loss is very low. However, I need to execute a query after each operation. This may affect performance. Also, I need to handle exceptions on database update.Synchronizing data in batches
There is only one ready solution that I could find here, MapDB. I'm planning to try it but I haven't tried it yet. If there is a more reliable, optimized sink solution that also uses db instead of file, I would prefer to use it.
Any recommendation to solve this question?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
您需要 Map 还是 TreeMap ?
整理顺序与存储、访问相关还是两者都不相关。
对于 Hazelcast,数据丢失的可能性是可配置的。您可以设置具有所需弹性级别的集群。这与磁盘相同,如果您有一个磁盘发生故障,您就会丢失数据。如果您有两台且其中一台离线,您不会丢失数据。您可以根据所需的弹性级别分配硬件。建议的最小值为三。
(每秒 10,000 个也不用担心,1,000,000,000 个已经完成。同步到外部存储可以立即或批量同步)
免责声明,我在 Hazelcast 工作,但我认为你的问题更基本 - 你如何保留你的存储可用的。
简单来说,就是不要重启。
集群解决方案就是答案。如果您有多个节点,即使少数节点离线,整个服务仍会保持运行。
进行滚动弹跳。
如果您必须立即重新启动所有内容,那么重要的是您的服务能够以多快的速度恢复所有数据,以及当恢复完成 50% 时它会做什么(50% 的数据可见吗?)。仅当您拥有尚未配置弹性的集群解决方案时,才真正需要立即复制到其他地方。如果你已经解决了弹性问题,间歇性保存就可以了。
因此,配置您的存储,使其不会离线,使备份/恢复的解决方案选项变得更加容易。
Do you need a Map or a TreeMap ?
Is collating sequence relevant for storage, for access or neither.
For Hazelcast, the chance for data loss is configurable. You set up a cluster with the level of resilience you want. This is the same as with disk, if you have one disk and it fails, you lose data. If you have two and one goes offline, you don't lose data. You allocate hardware for the level of resilience you need. Three is the recommended minimum.
(10,000 per second isn't worrying either, 1,000,000,000 has been done. Sync to an external store can be immediate or in batches)
Disclaimer, I work for Hazelcast, but I think your question is more fundamental -- how do you keep your store available.
Simply, don't restart.
Clustered solutions are the answer here. If you have multiple nodes, the service as a whole stays running even if a few nodes go offline.
Do rolling bounces.
If you must restart everything at once, what matters is how quickly can your service bring all data back and what does it do when the restore is 50% done (is 50% data visible?). Immediate replication to elsewhere is only really necessary if you have a clustered solution that hasn't been configured for resilience. Saving intermittently is fine if you have solved resilience.
So, configure your storage so that it doesn't go offline, makes the solution options for backup/restore all the easier.