Erlang:我应该以什么监督树结束编写任务调度程序?

发布于 2024-10-25 23:03:16 字数 1406 浏览 1 评论 0原文

主要是出于教育目的,我尝试编写一个任务(任务是一个 open_port({spawn_executable, Command}))调度程序。

我最终得到的树就像

        supervisor
        |        |
scheduler        receiver
gen_event        gen_event
                     |
                supervisor
                     |
                dispatcher
                gen_server
                     |
                supervisor
                |    |   |
             task1  ... taskN

换句话说:

  1. 顶部主管启动调度程序和接收器并确保它们处于活动状态
  2. 接收器启动中间主管
  3. 中间主管启动调度程序并确保其处于活动状态
  4. 调度程序启动底部主管
  5. 底部主管启动任务根据请求并确保在发生错误时重新启动

  6. 调度程序随时准备好接受带有时间戳的任务 执行

  7. 它应该在满足时间戳时执行它通知某些event_manager
  8. 接收器然后由同一事件管理器通知并通过消息传递给调度程序中间主管
  9. 调度程序有一些业务逻辑,这就是为什么它不是无状态的,例如,
  10. 当满足所有条件时,某些类型的任务不能同时执行,调度程序将任务传递给底层主管,这确保任务执行直到正常退出或某些阈值被绕过
  11. 底部主管返回一条消息,然后将其向上传递到某个事件管理器
  12. ,调度程序最终收到此消息,从其队列中删除任务或将其重新排队或执行其他操作

问题是:

  1. 我使用的行为正确吗?
  2. 结构是不是太复杂了? (但是,将来系统将变得分布式。)
  3. 有没有一种方法可以将接收者+中层管理器和调度器+底层管理器组合在两个模块中,而不是四个模块同时实现4种行为?
  4. 或者有没有一种方法将接收器+调度器+底层supervisor合并在一个模块中,消除中间supervisor的需要,同时实现gen_event+gen_server+supervisor的行为?
  5. 我是否错误地将行为视为面向对象语言中的接口或多重继承? (这让我问问题 3 和 4。)

提前致谢。

PS IMO,一方面,结构太复杂;另一方面,这样的结构允许我将其任何块进行分布式(例如,一个接收器的许多调度程序,一个接收器的一个调度程序,许多接收器的许多调度程序,每个接收器的许多调度程序,甚至每个调度程序的许多底层管理程序) - 每层都有自己的监督政策)。复杂性和可扩展性之间的平衡点在哪里?

Mostly in educational purposes I'm trying to write a task (task is an open_port({spawn_executable, Command})) scheduler.

I end up with the tree like

        supervisor
        |        |
scheduler        receiver
gen_event        gen_event
                     |
                supervisor
                     |
                dispatcher
                gen_server
                     |
                supervisor
                |    |   |
             task1  ... taskN

In other words:

  1. top supervisor starts scheduler and receiver and makes sure they will be alive
  2. receiver starts middle supervisor
  3. middle supervisor starts dispatcher and makes sure it will be alive
  4. dispatcher starts bottom supervisor
  5. bottom supervisor starts tasks upon request and makes sure they are restarted in case of error

  6. at any time scheduler is ready to accept a task with a timestamp it should be executed at

  7. when timestamp is met it notifies some event_manager
  8. receiver is then notified by the same event manager and passes the message to dispatcher through the middle supervisor
  9. dispatcher has some business logic that is why it is not stateless, for example, some kind of tasks cannot be executed simultaneously
  10. when all conditions are met dispatcher passes task to bottom supervisor which makes sure task is executed until normal exit is got or some thresold is bypassed
  11. bottom supervisor returns back a message which is then passed up-up-up to some event manager
  12. and scheduler eventually receives this message, removing task from its queue or reenqueueing it or something else

The questions are:

  1. Am I using behaviours right?
  2. Isn't the structure too complicated? (However, in future the system is going to become distributed.)
  3. Is there a way to combine receiver+middle supervisor and dispatcher+bottom supervisor in two modules instead of four implementing 4 behaviours in the same time?
  4. Or is there a way to combine receiver+dispatcher+bottom supervisor in one module, eliminating the need for middle supervisor, implementing gen_event+gen_server+supervisor behaviour at the same time?
  5. Am I mistaken thinking of behaviours as of interfaces or multi-inheritance in OO languages? (That makes me ask questions 3 and 4.)

Thanks in advance.

P. S. IMO, on one hand, the structure is too complicated; on the other hand such a structure lets me make any of its blocks distributed (for example, many schedulers to one receiver, one scheduler to many receivers, many schedulers to many receivers, many dispatchers for each receiver and even many bottom supervisors for each dispatcher - every layer with is own supervision policy). Where is the balance point between complexity and extensibility?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

哎呦我呸! 2024-11-01 23:03:16

我建议简化您的设计,就像:

        supervisor
        |        |
 dispatcher      |
 +scheduler      |
                 |
            supervisor
            |    |   |
         task1  ... taskN

即使从分布的角度来看,使用单独的调度程序将事件发送到启动任务等的调度程序也没有太多好处。

调度程序可以在计时器模块的帮助下非常简单地完成,并且可以是 gen_server。 Timer 可以发送您可以在handle_info 回调中处理的消息,也可以调用gen_server 的api 函数。

您还可以使用超时功能在下一个间隔后唤醒 gen_server,这会更简单,因为您不必担心在添加新“任务”时取消计时器。

然后调度程序/调度程序调用supervisor:start_child来添加工作任务。

可以轻松添加分发:调度程序/调度程序可以位于与第二级管理程序不同的节点上。任务启动功能可以进一步分发,并且可能使用 pool 模块进行负载平衡。

回答您的五个问题:

  1. 我怀疑您在不需要的地方使用 gen_event,但由于不需要模块本身,因此可以通过删除它们轻松修复。 gen_event 是,如果您希望能够在一个事件源上注册多个处理程序,则可以 1:1 的方式使用它。监督树通常是由作为其他监督者的直接子级的监督者构建的。

  2. 是的,它太复杂了,看起来有点像用表达能力较差的面向对象语言来做。仅仅为了准备可能的发行版是没有必要的。使用像 Erlang 这样的函数式语言进行重构比您想象的要容易得多。因此,如果您看到需要,请开始简单和拆分功能。

3+4。请参阅我完全不同的建议。

  1. 它不太像 OO。 OTP 中的行为只是回调模块,将流程机制隐藏在通用模块中。

即使是简单的结构,我建议也有足够的灵活性(由 Erlang 为您带来),因为如果您想要多个调度程序,您只需使用 rpc 来调用主管即可。您可以使用 pool 来自动负载平衡任务的分配。并且调度程序部分可以轻松地与调度程序分离(两者都在顶级主管之下),您可以将更常见的状态与调度程序分离。

What I would suggest is simplifying your design much more like:

        supervisor
        |        |
 dispatcher      |
 +scheduler      |
                 |
            supervisor
            |    |   |
         task1  ... taskN

The there is not much gain from having a separate scheduler sending events to a dispatcher which starts tasks etc. Even in the light of distribution.

The dispatcher-scheduler can be probably quite simply done with the help of the timer module and can be a gen_server. Timer can either send messages which you can process in the handle_info callback or call api functions of your gen_server.

You could also use the timeout functionality to wake up the gen_server after the next interval that would be even simpler since you don't have to worry abou canceling timers when you add a new "task".

The dispatcher/scheduler then calls supervisor:start_child to add working tasks.

Distribution can be added easily: dispatcher/scheduler can be on a separate node than the second level supervisor. The tasks starting function can distribute further and maybe using the pool module for load balancing.

To answer your five questions:

  1. I suspect you are using gen_event where it is not needed, but since the modules itself are not needed its easily fixed by removing them. gen_event is if you want to be able to register many handlers on one event source, you are using it 1:1. Supervision trees are usually built with supervisors being the direct child of other supervisors.

  2. Yes its too complicated, looks a bit like you would do it in OO languages with less expressive power. And just to prepare for a maybe distribution its not necessary. Refactoring in a functional language like Erlang is much easier than you probably think. So start simple and split functionality if you see the need.

3+4. See my altogether different suggestion.

  1. Its not ver OO like. Behaviours in OTP are only callback modules hiding the process mechanics in a generic module.

Even with the simple structure I suggested there is plenty of flexibility (brought to you by Erlang) because if you want to have multiple schedulers you can just use rpc to call the supervisor. You can use pool to automatically load balance distribution of tasks. And the dispatcher part can be easily separated from the scheduler (both under the toplevel supervisor then) the you can have more common state separated from the scheduler.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文