用于建模作业执行流程的设计模式

发布于 2024-12-27 03:30:26 字数 360 浏览 1 评论 0原文

在我的应用程序中,我有一组要执行的作业。每个作业都会经历“未开始”、“已开始”、“已完成”、“失败”等状态。每个作业都有一组前置条件和后置条件。在满足先决条件之前,作业无法启动,如果不满足后置条件,则应将其标记为失败。

例如,假设作业将文本文件导入数据库。前置条件是检查源文件是否存在,后置条件是检查数据库中是否存在数据。

除了这些前置和后置条件之外,有时一项工作还依赖于其他工作来完成。创建作业表并拥有作业的依赖关系表很容易,但实际上是否可以使这些验证前和验证后检查在数据库中可配置(这样,如果这些条件发生变化或添加了新条件)?即使有可能,这样做是个好主意吗?

需要使该模型通用,以便其他应用程序也可以使用它,即使要执行的验证检查对于其他应用程序完全不同。

In my application I have a set of jobs to execute. Each jobs goes through the states "not started", "started", "completed", "failed" etc. Each job has a set of preconditions and post-conditions. A job cannot start until the preconditions are satisfied and should be marked as failed if it doesn't satisfy the post conditions.

For example, let's say the job imports a text file into the database. The precondition would be to check if the source file exists and post condition would be to check if data exists in database.

On top of these pre and post conditions, sometimes a job is also dependent on other jobs to finish. It is easy to create a jobs table and have a dependency table for jobs, but is it actually possible to make these pre and post validation checks to be configurable in the database (so that no code changes need to be made if these conditions change or new conditions are added)? Even if it is possible somehow, is it a good idea to do so?

There is a requirement to make this model generic so that other applications can also make use of it even if the validation checks to be performed are entirely different for other applications.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

↙温凉少女 2025-01-03 03:30:26

我认为你冒着尝试过多驾驶的风险。通过尝试表驱动所有验证前和验证后条件,您正接近尝试在数据中编写代码的危险。

我已经构建了一些非常复杂的作业调度应用程序。其中一个可能特别令人感兴趣的是每日 ETL 流程,该流程根据平面文件源加载数十个 SQL 表。

现有系统使用线性过程,程序员必须手动计算表间依赖关系并按给定顺序运行表加载。这样做的问题是,如果任何进程失败,其余的作业就必须坐下来等待问题解决。

我构建了一个新系统,该系统具有表驱动的元数据,可以指出直接的表间依赖关系。换句话说,表 A 具有表 B 和表 C 的 FK。无需手动跟踪所有相互依赖关系,而是仅跟踪直接相互依赖关系。然后调度程序只需查看哪些加载已完成,哪些加载尚未完成。任何没有不完整依赖关系的挂起负载都可以启动。

我认为你应该类似地构建你的系统。使用关注点分离。不要表驱动依赖项是什么,而应该只表驱动存在哪些依赖项。您可以在调度表中跟踪这些依赖项中哪些已通过,哪些已失败。数据库不需要知道如何进行这些测试。让代码关心依赖项到底是什么以及如何测试它们是通过还是失败。这就是您的作业调度程序需要知道的全部内容。避免创建源代码位于数据库中的脚本语言的诱惑。

I think you run the risk of trying to table drive too much. By attempting to table drive all of the pre and post validation conditions you are getting dangerously close to trying to write code in data.

I have built some pretty sophisticated job scheduling applications. One in particular that might be of interest was a daily ETL process that loaded dozens of SQL tables based on flat file feeds.

The existing system used a linear process where the programmer had to manually work out the inter-table dependencies and run the table loads in a given order. The problem with this was that if any process failed, the rest of the jobs had to sit and wait until the problem was resolved.

I built a new system that had table driven metadata that pointed out the immediate inter-table dependencies. In other words, table A has FKs to tables B and C. Instead of manually keeping track of all of the interdependencies manually, only the immediate interdependencies were tracked. Then the scheduler just had to look at which loads had completed and which loads hadn't. Any pending load which had no incomplete dependencies was OK to start.

I think you should build your system similarly. Use separation of concerns. Don't table drive what the dependencies are, instead you should just table drive which dependencies exist. You can track in your scheduling tables which of these dependencies have passed and which have failed. The database doesn't need to know how to do these tests. Let the code worry about what the dependencies are exactly and how to test whether they pass or fail. That is all your job scheduler needs to know. Avoid the temptation to create a scripting language whose source code sits in your database.

活泼老夫 2025-01-03 03:30:26

考虑将您的应用程序与规则引擎(也称为业务规则)集成。这个想法是逻辑在代码外部定义并存储在表或文件中。规则引擎读取规则并进行解释。

这些软件很快就会变得复杂。大多数情况下,它们是商业包,但也存在一些免费和开源框架。我没有使用过任何免费套餐。一般来说,我建议研究现有代码,而不是从头开始构建规则引擎。

马丁·福勒的精彩介绍:
http://martinfowler.com/bliki/RulesEngine.html

一篇内容更丰富的文章:
http://www.infoq.com/articles/Rule-Engines

查找实际代码,Google 搜索“规则引擎”或“工作流引擎”,然后添加您的编程语言的名称(“Java”、“C#”或其他语言)。

Consider integrating your application with a rules engine (also know as business rules). The idea is logic is defined outside of the code and stored in a table or file. The rule engine reads the rule and interprets.

These pieces of software can get complicated pretty quickly. Mostly, they are commercial packages, but some free and open-source frameworks exist. I have not used any of the free packages. In general, I recommend looking into exiting code rather than building a rules engine from scratch.

Nice intro by Martin Fowler:
http://martinfowler.com/bliki/RulesEngine.html

An article with a little more substance:
http://www.infoq.com/articles/Rule-Engines

To find actual code, Google on "rules engine" or "workflow engine" and add in the name of your programming language ("Java", "C#", or what-not).

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文