在多个服务器上批量运行

发布于 2024-11-15 08:41:31 字数 375 浏览 3 评论 0原文

我有一个批处理应用程序,仅包含以下步骤。

  • 从表中选择 update-flag 列为空的所有记录
  • 处理所选信息
  • 将记录标记为已更新

我正在使用 Oracle 10g,我想要做的是能够在多个服务器上同时执行此应用程序。不过,我担心Oracle的锁机制。 当在第一台服务器上运行的应用程序的第一个实例选择一条记录时,在第二台服务器上运行的另一个实例不应该能够选择它。

是否适合使用“选择更新”来达到这种目的?如果我使用“选择更新跳过锁定”模式,应用程序的第二个实例是否会默默地跳过先前由第一个实例锁定的那些记录并选择其他行?或者,它会等到锁定的行被释放吗?

任何意见和建议将不胜感激

I have a batch application which simply consists of these steps.

  • Select all the records from the table where the update-flag column is null
  • Process the selected information
  • Mark the record as updated

I'm using Oracle 10g and what I'm trying to do is to be able to execute this application on multiple servers concurrently. However, I'm concerned about the lock mechanism of Oracle.
When the first instance of my application running on the first server selects a record, the other instance running on the second server should not be able to select it.

Is it suitable to use "select for update" for this kind of purpose? If I use "select for update skip locked" pattern, does the second instance of the application silently skip those records locked by the first instance previously and select those other rows? Or, does it wait until the locked rows are released?

Any advice and suggestions will be greatly appreciated

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

山田美奈子 2024-11-22 08:41:31

我的同事 Rob 写了一篇关于此的好文章 skip 中的并行性锁定场景
所以是的,它有效,不,它不会帮助你。
FWIW:稍微改变算法可能更聪明,给更新标志一个值,一个增量数字来帮助处理新数据的选择过程,并在准备好时更新为 NULL。这里的优点是

  1. 索引大小减少了
  2. 两倍,这使得并行性更容易实现,因为进程现在都可以选择它们“自己的”行。

我希望它有帮助。

My collegue Rob wrote a nice article about this Parallellism in a skip locked scenario
So yes, it works, no, it wont help you.
FWIW: it might be smarter to change the algoritm a little, give the update flag a value, an incremental number to help the selection processes that handle the new data, and update to NULL when ready. The advantage here is two fold

  1. reduces the index size
  2. it makes parallism easier to implement because processes can now all select ther 'own' rows.

I hope it helps.

深陷 2024-11-22 08:41:31

我使用选择更新跳过锁定并且工作正常。是的,后面的会话会默默地跳过先前会话锁定的记录并选择/锁定其他行。确保并行处理会话没有修改其他常见数据,否则您会陷入另一个瓶颈。

declare 
  my_limit constant number default 1000;
  cursor cRecords is
    select primary_key
    from processed_table t
    where t.update_flag is null
    for update skip locked;
  type t_cRecords is table of cRecords%rowtype;
  tRecords t_cRecords;
begin
  open cRecords;
  while true loop
    -- Select all the records from the table where the update-flag column is null
    fetch cRecords bulk collect into tRecords limit my_limit;
    -- Process the selected information
    -- ...
    -- Mark the record as updated
    forall i in tRecords.first..tRecords.last
      update processed_table
        set update_flag = 'PROCESSED'
      where primary_key = tRecords(i).primary_key;
    --
    exit when tRecords.count < my_limit;
  end loop;
end;

顺便说一句,我不确认 skip 中的并行性锁定场景 - 在我的网站上,使用 rwijk 的脚本,速度加快了 4 到 1 秒。

I use select for update skip locked and it works fine. Yes, later sessions silently skip records locked by earlier sessions and select/lock other rows. Be sure there is no other common data modified by parallel processing sessions or you get stuck in another bottleneck.

declare 
  my_limit constant number default 1000;
  cursor cRecords is
    select primary_key
    from processed_table t
    where t.update_flag is null
    for update skip locked;
  type t_cRecords is table of cRecords%rowtype;
  tRecords t_cRecords;
begin
  open cRecords;
  while true loop
    -- Select all the records from the table where the update-flag column is null
    fetch cRecords bulk collect into tRecords limit my_limit;
    -- Process the selected information
    -- ...
    -- Mark the record as updated
    forall i in tRecords.first..tRecords.last
      update processed_table
        set update_flag = 'PROCESSED'
      where primary_key = tRecords(i).primary_key;
    --
    exit when tRecords.count < my_limit;
  end loop;
end;

By the way, I don't confirm Parallellism in a skip locked scenario - on my site there is from 4 to one second speed up, using scripts from rwijk.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文