为什么对于具有父子约束表的大型数据集,PL/SQL Bulk DML 运行速度会变慢?

发布于 2024-10-21 16:10:59 字数 2884 浏览 1 评论 0 原文

我一直试图弄清楚为什么这个 PL/SQL 清除脚本对于记录表有几十万或更多记录的数据集运行缓慢。在脚本执行之前,记录表的某个子集被标记为要清除 - 大约 75%。

是什么导致删除 Record_Part 表比其他表花费更长的时间?是因为它位于3表父子层次结构的中间吗?我是否在索引或约束方面遗漏了一些知识?我可以做什么来加速这个定期清除过程?

这是一个 Oracle 10g 数据库。

预先感谢您阅读我的问题。

架构(部分):

  • Record表是父表
  • Record_Part表是Record的子表(Record有许多Record_Part)
  • Record_Person是Record_Part的子表(Record_Part有许多Record_Person)
  • 典型比例为1:7 :9 (record:record_part:record_person)

记录

  • PK - sysid
  • 物理 ID
  • 待处理
  • purge_in_progress

Record_Part

  • PK - Part_pk
  • FK - record_sysid

Record_Person

  • PK - sysid
  • FK - Part_pk

运行时

50000 个记录条目

  • record_person forall 在 1:40 分钟内完成
  • record_part forall 在 1 分 20 分钟内完成
  • record forall 在 10 秒内完成

300000 个记录条目

  • record_person forall 在 9 分钟内完成
  • record_part forall 在 2 小时内完成
  • record forall 在 20 分钟内完成

2000000 个记录条目

  • record_person forall 在 1 内完成 小时
  • record_part forall 在 13 小时内完成 小时 (!)
  • 记录在 8 分钟内完成

索引和约束 DDL

alter table Record add constraint record_REC_PK primary key (SYSID) using index tablespace DB_INDEX1;
alter table Record_Part add constraint RECPART_REC_PK primary key (Part_PK) using index tablespace DB_INDEX1;
alter table Record_Part add constraint RECPART_FK foreign key (RECORD_SYSID) references record (SYSID);
alter table Record_Person add constraint RECPERSON_REC_PK primary key (SYSID) using index tablespace DB_INDEX1;
alter table Record_Person add constraint RECPERSON_FK foreign key (Part_PK) references Record_Part (Part_PK);

CREATE INDEX REC_PURGE_IDX ON record (PURGE_IN_PROGRESS);
CREATE INDEX REC_PHYSID_IDX ON record (PHYSICALID);
CREATE INDEX REC_PENDING_IDX ON record (PENDING);
CREATE INDEX RECPART_RECORD_SYSID_IDX ON Record_Part (RECORD_SYSID);
CREATE INDEX RECPERSON_PARTPK_IDX on Record_Person (PART_PK);

脚本: (下面的脚本中省略了时间戳打印)

DECLARE

TYPE sSysid IS TABLE OF record.sysid%TYPE
    INDEX BY PLS_INTEGER;

TYPE physicalid IS TABLE OF record.physicalid%TYPE
    INDEX BY PLS_INTEGER;    

l_sid sSysid;
l_physicalid physicalid;

BEGIN
    SELECT sysid, physicalid
    BULK COLLECT INTO l_sid, l_physicalid
        FROM record
        where purge_in_progress = 1;

FORALL i IN l_sid.FIRST .. l_sid.LAST
    delete from record_person where Part_pk like concat(l_sid(i), '%') or Part_pk like concat(l_physicalid(i), '%');

commit;

FORALL i IN l_sid.FIRST .. l_sid.LAST
    delete from record_Part where record_sysid = l_sid(i);

commit;

FORALL i IN l_sid.FIRST .. l_sid.LAST
    delete from record where sysid = l_sid(i);

END;
/

commit;

I have been trying to figure out why this PL/SQL purge script runs slowly for datasets where the record table has a a few hundred thousand or more records. Prior to script execution, a certain subset of the Record table is flagged for purging - about 75%.

What causes the deletion of the Record_Part table to take so much longer than the other tables? Is it because it is in the middle of the 3-table parent-child hierarchy? Am I missing some piece of knowledge here in terms of indexes or constraints? What can I do to speed up this periodic purging process?

This is an Oracle 10g database.

Thanks in advance for reading my question.

Schema (Partial):

  • Record table is the parent table
  • Record_Part table is the child of Record (Record has many Record_Part)
  • Record_Person is the child of Record_Part (Record_Part has many Record_Person)
  • The typical ratio is 1:7:9 (record:record_part:record_person)

Record

  • PK - sysid
  • physicalid
  • pending
  • purge_in_progress

Record_Part

  • PK - Part_pk
  • FK - record_sysid

Record_Person

  • PK - sysid
  • FK - Part_pk

Runtimes

50000 record entries

  • record_person forall completes in 1:40 min
  • record_part forall completes in 1:20 min
  • record forall completes in 10 seconds

300000 record entries

  • record_person forall completes in 9 min
  • record_part forall completes in 2 hours
  • record forall completes in 20 minutes

2000000 record entries

  • record_person forall completes in 1
    hour
  • record_part forall completes in 13
    hours (!)
  • record forall completes in 8 minutes

Index and Constraint DDL

alter table Record add constraint record_REC_PK primary key (SYSID) using index tablespace DB_INDEX1;
alter table Record_Part add constraint RECPART_REC_PK primary key (Part_PK) using index tablespace DB_INDEX1;
alter table Record_Part add constraint RECPART_FK foreign key (RECORD_SYSID) references record (SYSID);
alter table Record_Person add constraint RECPERSON_REC_PK primary key (SYSID) using index tablespace DB_INDEX1;
alter table Record_Person add constraint RECPERSON_FK foreign key (Part_PK) references Record_Part (Part_PK);

CREATE INDEX REC_PURGE_IDX ON record (PURGE_IN_PROGRESS);
CREATE INDEX REC_PHYSID_IDX ON record (PHYSICALID);
CREATE INDEX REC_PENDING_IDX ON record (PENDING);
CREATE INDEX RECPART_RECORD_SYSID_IDX ON Record_Part (RECORD_SYSID);
CREATE INDEX RECPERSON_PARTPK_IDX on Record_Person (PART_PK);

Script:
(timestamp printing omitted from script below)

DECLARE

TYPE sSysid IS TABLE OF record.sysid%TYPE
    INDEX BY PLS_INTEGER;

TYPE physicalid IS TABLE OF record.physicalid%TYPE
    INDEX BY PLS_INTEGER;    

l_sid sSysid;
l_physicalid physicalid;

BEGIN
    SELECT sysid, physicalid
    BULK COLLECT INTO l_sid, l_physicalid
        FROM record
        where purge_in_progress = 1;

FORALL i IN l_sid.FIRST .. l_sid.LAST
    delete from record_person where Part_pk like concat(l_sid(i), '%') or Part_pk like concat(l_physicalid(i), '%');

commit;

FORALL i IN l_sid.FIRST .. l_sid.LAST
    delete from record_Part where record_sysid = l_sid(i);

commit;

FORALL i IN l_sid.FIRST .. l_sid.LAST
    delete from record where sysid = l_sid(i);

END;
/

commit;

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

凉风有信 2024-10-28 16:10:59

检查此线程的第一个回复。正如 Justin 指出的,您需要使用 limit 子句来获取固定数量的记录(通常使用 100 条,您可以对其进行参数化,看看什么适合您的情况)。

http://asktom.oracle。 com/pls/asktom/f?p=100:11:0::::p11_question_id:5918938803188

Check the first reply on this thread. As Justin pointed out, you need to use the limit clause to fetch a fixed number of records (100 is the generally used, you can parameterize this and see what works for your situation).

http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::p11_question_id:5918938803188

め可乐爱微笑 2024-10-28 16:10:59

您可以在测试环境中禁用 FK 约束来看看是否有帮助?

另一种可能性是将 FK 约束重新创建为可延迟的,并在脚本开始时延迟约束,例如:

alter table Record_Part 
      add constraint RECPART_FK foreign key (RECORD_SYSID) 
                                references record (SYSID) DEFERRABLE;
alter table Record_Person 
      add constraint RECPERSON_FK foreign key (Part_PK) 
                                  references Record_Part (Part_PK) DEFERRABLE;

SET CONSTRAINTS ALL DEFERRED;
...run your purge
SET CONSTRAINTS ALL IMMEDIATE;

请注意,任何 COMMIT 将导致立即设置约束。每次提交后,您必须重新发出第一个设置约束语句。

限制因素将是我的第一个怀疑。

Can you disable the FK constraints in a test environment to see if that helps?

Another possibility is to re-create the FK constraints as deferrable, and defer the constraints at the start of the script, e.g.:

alter table Record_Part 
      add constraint RECPART_FK foreign key (RECORD_SYSID) 
                                references record (SYSID) DEFERRABLE;
alter table Record_Person 
      add constraint RECPERSON_FK foreign key (Part_PK) 
                                  references Record_Part (Part_PK) DEFERRABLE;

SET CONSTRAINTS ALL DEFERRED;
...run your purge
SET CONSTRAINTS ALL IMMEDIATE;

Note that any COMMIT will cause the constraints to be set immediate. You will have to reissue the first set constraints statement after each commit.

The constraints would be my first suspect here.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文