需要 24 x 5 运行时处理日常批处理作业的策略
我们正在尝试将一个应用程序迁移到 24x5 运行。我们有一些批处理作业，可以在夜间导入数据源以更新第二天的客户更改。该作业的结构是擦除并重建整个表的数百万行。在此作业期间，该表大约有 2 小时不可用。
它足够大，以至于在更新之前加载到内存中是不可行的。我想知道在批处理作业完成之前以其他方式提供数据的最佳方法是什么。
以 xml 格式将表加载到磁盘？ 在批处理作业之前复制表？
我们有一个灾难恢复堆栈，但我认为通过其设置方式，它将尝试实时同步，因此在作业执行期间它也将不可用。
数据库是DB2，前端是IBM Websphere
We have an application that we are attempting to move to a 24x5 operation. We have a few batch jobs that import data feeds overnight to update customer changes for the following day. This job is structured that the entire table several million rows is wiped and rebuilt. During this job the table is unavailable for about 2 hours.
Its large enough that its not feasible to load into memory before updating. I wanted to know what the best approach is to making the data available some other way until the batch job is finished.
Loading the table to disk in xml format?
Copying the table before the batch job?
We have a DR stack but I think with the way its setup it will try and sync realtime so it will be unavailable as well during the job execution.
Database is DB2, front end is IBM Websphere
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
评估杀戮和填充策略的必要性。检查是否可以将其更改为仅应用适当的增量来进行添加和更改，然后就可以了。
您给数据库带来了巨大的负载来释放和重新捕获页面。
如果您可以执行增量操作，但只需要保留几天的数据，则可以在表上使用分区，并在分区超出范围时删除分区，而不必删除记录。
如果您必须进行 Kill-And-Fill，请使用原始表（我们将其称为
BatchTable）创建一个具有相同结构的名为BatchTable_Process的新表。使用BatchTable_Process处理批处理作业数据。处理完成后，交换表名。Evaluate the need for the kill-and-fill strategy. Check if you can change that to just apply the appropriate deltas for the additions and changes and get away with it.
You are putting a huge load on the database to release and recapture pages.
If you can do a delta, but only need to keep a few days worth of data, you can use partitioning on the table and drop partitions as they go out of scope instead of having to drop records.
If you must go for the Kill-And-Fill, use your original table (lets call it
BatchTable) to create a new table calledBatchTable_Processwith the same structure. UseBatchTable_Processto process the batch job data. Once processing is done, swap table names.