使用 Java 归档数据库表
我的应用程序要求使用 java 在 sybase 和 db2 之间归档数据库表，反之亦然，以及在（db2 到 db2 和 sybase 到 sybase）内归档数据库表。
我试图了解性能、实施、易用性和可扩展性方面的最佳策略。
这是我当前的流程 -
- 具有可接受参数（来自 java）的源表和目标表在 xml 中定义。 [实际查询被放置在 xml 中，因为有时从 java 接受参数（例如，对于 where 子句条件）]
- 应用程序读取源和目标配置并按顺序执行它们。
- 当源只是从特定表中删除数据或源只是调用存储过程时，目标有时是可选的。
- 源和目标之间的数据集非常大（以百万计）
从我的想法来看，我似乎可以定义多个源和目标组合之间的依赖关系，并让它们在多个步骤中并行执行。但这会提高性能吗（我希望如此）？
有没有使用java进行数据归档的开源框架？关于工具方面的任何其他想法都会非常有帮助。
谢谢
My application demands archiving database tables between sybase and db2 and vice-a-versa and within(db2 to db2 and sybase to sybase) using java.
I am trying to understand the best strategies around in terms of performance, implementation, ease of use and scalability.
Here is my current process -
- source and destination tables with the acceptable parameters (from java) are defined within xml. [actual query is being placed inside the xml because at sometimes parameters are accepted from java (for a where clause condition for example)]
- the application reads the source and destination configurations and execute them sequentially.
- destination is sometimes optional when source is just deleting data from a specific table or when the source is just calling a stored procedure.
- dataset between source and destination is extremely large (in millions)
From top of my head, it looks like I can define dependencies between multiple source and destination combination and have them execute in parallel in multiple treads. But will this improve any performance(i hope it will)?
Are there any open-source frameworks for data archiving using java? Any other thoughts on the implements side will be really helpful.
Thanks
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
最强大的 Java 持久性开源框架是 Hibernate。您可以从现有数据库对 Java 模型进行逆向工程（请参阅 Hibernate 工具），并使用 Session.replicate() 执行复制。您可以通过使用无状态会话和二级缓存（如果适用）来微调性能。文档位于此处
The most powerful open source framework for Java persistence is Hibernate. You can reverse engineer Java model from existing DB (see Hibernate Tools), and perform a replication using Session.replicate(). You can fine tune performance by using stateless sessions and second level caching where applicable. Documentation is here
查看一些数据库复制工具（我们使用 Shadowbase）。他们可能有 Java API。
另外，请查看此 IBM 白皮书：
Look at some database replication tools (we use Shadowbase). They might have Java API's.
Also, check out this IBM whitepaper:
Pentaho Data Integration 对在数据库之间复制数据或从数据库复制数据具有强大的支持。另外，它是开源的，允许您用 Java 编写插件。
从 Oracle 迁移到 MySQL
Pentaho Data Integration has robust support for copying data between or from databases. Plus, it's Open Source and allows you write plugins in Java.
Migrate from Oracle to MySQL
您需要做的最重要的事情是禁用 JDBC 中的自动提交，否则您将在每次插入数据库表后进行提交。这会破坏性能。
但您基本上必须弄清楚您的同步方案，以便您可以确定需要复制哪些记录，然后才能决定如何实际执行。
The single most important thing you need to do is to disable auto-commit in JDBC, as you would otherwise commit after each insert in the database table. That ruins performance.
But you basically have to figure out your synchronization scheme so you can identify which records need to be copied before you can decide on how to actually do it.