需要将递归 CTE 查询转换为索引友好查询

发布于 2024-10-14 10:20:59 字数 1891 浏览 7 评论 0原文

在经历了编写递归 CTE 查询以满足我的需求的所有艰苦工作之后,我意识到我无法使用它,因为它在索引视图中不起作用。所以我需要其他东西来代替下面的 CTE。 (是的,您可以在非索引视图中使用 CTE,但这对我来说太慢了)。

要求:

  1. 我的最终目标是拥有一个自我更新的索引视图(它不一定是视图,而是类似的东西)......也就是说,如果数据发生任何变化视图连接的表的数量,则视图需要自行更新。

  2. 视图需要建立索引,因为它必须非常快,并且数据不会经常更改。不幸的是,使用 CTE 的非索引视图需要 3-5 秒才能运行,这对于我的需求来说太长了。我需要查询以毫秒为单位运行。递归表中有几十万条记录。

就我的研究而言,满足所有这些要求的最佳解决方案是索引视图,但我对任何解决方案都持开放态度。

CTE 可以在我的 其他帖子的答案中找到。 或者又是这样:

DECLARE @tbl TABLE ( 
     Id INT 
    ,[Name] VARCHAR(20) 
    ,ParentId INT 
    ) 

INSERT INTO @tbl( Id, Name, ParentId ) 
VALUES 
 (1, 'Europe', NULL) 
,(2, 'Asia',   NULL) 
,(3, 'Germany', 1) 
,(4, 'UK',      1) 
,(5, 'China',   2) 
,(6, 'India',   2) 
,(7, 'Scotland', 4) 
,(8, 'Edinburgh', 7) 
,(9, 'Leith', 8) 

; 
DECLARE @tbl2 table (id int, abbreviation varchar(10), tbl_id int)
INSERT INTO @tbl2( Id, Abbreviation, tbl_id ) 
VALUES 
 (100, 'EU', 1) 
,(101, 'AS', 2) 
,(102, 'DE', 3) 
,(103, 'CN', 5)

;WITH abbr AS (
    SELECT a.*, isnull(b.abbreviation,'') abbreviation
    FROM @tbl a
    left join @tbl2 b on a.Id = b.tbl_id
), abcd AS ( 
          -- anchor 
        SELECT  id, [Name], ParentID,
                CAST(([Name]) AS VARCHAR(1000)) [Path],
                cast(abbreviation as varchar(max)) abbreviation
        FROM    abbr
        WHERE   ParentId IS NULL 
        UNION ALL
          --recursive member 
        SELECT  t.id, t.[Name], t.ParentID, 
                CAST((a.path + '/' + t.Name) AS VARCHAR(1000)) [Path],
                isnull(nullif(t.abbreviation,'')+',', '') + a.abbreviation
        FROM    abbr AS t 
                JOIN abcd AS a 
                  ON t.ParentId = a.id 
       )
SELECT *, [Path] + ':' + abbreviation
FROM abcd

After going through all the hard work of writing a recursive CTE query to meet my needs, I realize I can't use it because it doesn't work in an indexed view. So I need something else to replace the CTE below. (Yes you can use a CTE in a non-indexed view, but that's too slow for me).

The requirements:

  1. My ultimate goal is to have a self updating indexed view (it doesn't have to be a view, but something similar)... that is, if data changes in any of the tables the view joins on, then the view needs to update itself.

  2. The view needs to be indexed because it has to be very fast, and the data doesn't change very frequently. Unfortunately, the non-indexed view using a CTE takes 3-5 seconds to run which is way too long for my needs. I need the query to run in milliseconds. The recursive table has a few hundred thousand records in it.

As far as my research has taken me, the best solution to meet all these requirements is an indexed view, but I'm open to any solution.

The CTE can be found in the answer to my other post.
Or here it is again:

DECLARE @tbl TABLE ( 
     Id INT 
    ,[Name] VARCHAR(20) 
    ,ParentId INT 
    ) 

INSERT INTO @tbl( Id, Name, ParentId ) 
VALUES 
 (1, 'Europe', NULL) 
,(2, 'Asia',   NULL) 
,(3, 'Germany', 1) 
,(4, 'UK',      1) 
,(5, 'China',   2) 
,(6, 'India',   2) 
,(7, 'Scotland', 4) 
,(8, 'Edinburgh', 7) 
,(9, 'Leith', 8) 

; 
DECLARE @tbl2 table (id int, abbreviation varchar(10), tbl_id int)
INSERT INTO @tbl2( Id, Abbreviation, tbl_id ) 
VALUES 
 (100, 'EU', 1) 
,(101, 'AS', 2) 
,(102, 'DE', 3) 
,(103, 'CN', 5)

;WITH abbr AS (
    SELECT a.*, isnull(b.abbreviation,'') abbreviation
    FROM @tbl a
    left join @tbl2 b on a.Id = b.tbl_id
), abcd AS ( 
          -- anchor 
        SELECT  id, [Name], ParentID,
                CAST(([Name]) AS VARCHAR(1000)) [Path],
                cast(abbreviation as varchar(max)) abbreviation
        FROM    abbr
        WHERE   ParentId IS NULL 
        UNION ALL
          --recursive member 
        SELECT  t.id, t.[Name], t.ParentID, 
                CAST((a.path + '/' + t.Name) AS VARCHAR(1000)) [Path],
                isnull(nullif(t.abbreviation,'')+',', '') + a.abbreviation
        FROM    abbr AS t 
                JOIN abcd AS a 
                  ON t.ParentId = a.id 
       )
SELECT *, [Path] + ':' + abbreviation
FROM abcd

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

耳根太软 2024-10-21 10:20:59

在克服了索引视图的所有障碍(自连接、cte、udf 访问数据等)之后,我建议将以下内容作为您的解决方案。

创建支撑函数

基于距根部的最大深度 4(总共 5 个)。或者使用 CTE

CREATE FUNCTION dbo.GetHierPath(@hier_id int) returns varchar(max)
WITH SCHEMABINDING
as
begin
return (
    select FullPath =
               isnull(H5.Name+'/','') + 
               isnull(H4.Name+'/','') +
               isnull(H3.Name+'/','') +
               isnull(H2.Name+'/','') +
               H1.Name
             +
               ':'
             +
               isnull(STUFF(
               isnull(','+A1.abbreviation,'') +
               isnull(','+A2.abbreviation,'') + 
               isnull(','+A3.abbreviation,'') +
               isnull(','+A4.abbreviation,'') +
               isnull(','+A5.abbreviation,''),1,1,''),'')
    from dbo.HIER H1
    left join dbo.ABBR A1 on A1.hier_id = H1.Id
    left join dbo.HIER H2 on H1.ParentId = H2.Id
    left join dbo.ABBR A2 on A2.hier_id = H2.Id
    left join dbo.HIER H3 on H2.ParentId = H3.Id
    left join dbo.ABBR A3 on A3.hier_id = H3.Id
    left join dbo.HIER H4 on H3.ParentId = H4.Id
    left join dbo.ABBR A4 on A4.hier_id = H4.Id
    left join dbo.HIER H5 on H4.ParentId = H5.Id
    left join dbo.ABBR A5 on A5.hier_id = H5.Id
    where H1.id = @hier_id)
end
GO

将列添加到表本身

例如完整路径列,如果需要,可以通过在 ':' (left=>path, right=>abbreviations)

-- index maximum key length is 900, based on your data, 400 is enough
ALTER TABLE HIER ADD FullPath VARCHAR(400)

维护列

由于层次性质,记录 X 可能会被删除,从而影响 Y 后代和 Z 祖先,这在 INSTEAD OF 或 AFTER 触发器中很难识别。因此,替代方法是基于以下条件:

  • 如果视图连接的任何表中的数据发生更改,则视图需要自行更新。
  • 使用 CTE 的非索引视图需要 3-5 秒才能运行,这对于我的需求来说太长了

我们只需再次运行整个表来维护数据,每次更新需要 3-5 秒(如果 5-5 秒则更快)连接查询效果更好)。

CREATE TRIGGER TG_HIER
ON HIER
AFTER INSERT, UPDATE, DELETE
AS
UPDATE HIER
SET FullPath = dbo.GetHierPath(HIER.Id)

最后,在表本身上为新列建立索引

create index ix_hier_fullpath on HIER(FullPath)

如果您打算通过 id 访问路径数据,那么它已经在表本身中,无需添加额外的索引。

上面的 TSQL 引用这些对象

修改表和列名称以适合您的架构。

CREATE TABLE dbo.HIER (Id INT Primary Key Clustered, [Name] VARCHAR(20) ,ParentId INT)
;
INSERT dbo.HIER( Id, Name, ParentId ) VALUES 
     (1, 'Europe', NULL) 
    ,(2, 'Asia',   NULL) 
    ,(3, 'Germany', 1) 
    ,(4, 'UK',      1) 
    ,(5, 'China',   2) 
    ,(6, 'India',   2) 
    ,(7, 'Scotland', 4) 
    ,(8, 'Edinburgh', 7) 
    ,(9, 'Leith', 8)
    ,(10, 'Antartica', NULL) 
; 
CREATE TABLE dbo.ABBR (id int primary key clustered, abbreviation varchar(10), hier_id int)
;
INSERT dbo.ABBR( Id, Abbreviation, hier_id ) VALUES 
     (100, 'EU', 1) 
    ,(101, 'AS', 2) 
    ,(102, 'DE', 3) 
    ,(103, 'CN', 5)
GO

编辑 - 可能更快的替代方案

鉴于每次都会重新计算所有记录,因此实际上不需要返回单个 HIER.ID 的 FullPath 的函数。 支持函数中的查询可以在末尾不使用where H1.id = @hier_id过滤器的情况下使用。此外,FullPath 的表达式可以轻松地从中间分解为 PathOnlyAbbreviation。或者只使用原始 CTE,以更快者为准。

After hitting all the roadblocks with indexed views (self join, cte, udf accessing data etc), I propose that the below as a solution for you.

Create support function

Based on maximum depth of 4 from root (5 total). Or use a CTE

CREATE FUNCTION dbo.GetHierPath(@hier_id int) returns varchar(max)
WITH SCHEMABINDING
as
begin
return (
    select FullPath =
               isnull(H5.Name+'/','') + 
               isnull(H4.Name+'/','') +
               isnull(H3.Name+'/','') +
               isnull(H2.Name+'/','') +
               H1.Name
             +
               ':'
             +
               isnull(STUFF(
               isnull(','+A1.abbreviation,'') +
               isnull(','+A2.abbreviation,'') + 
               isnull(','+A3.abbreviation,'') +
               isnull(','+A4.abbreviation,'') +
               isnull(','+A5.abbreviation,''),1,1,''),'')
    from dbo.HIER H1
    left join dbo.ABBR A1 on A1.hier_id = H1.Id
    left join dbo.HIER H2 on H1.ParentId = H2.Id
    left join dbo.ABBR A2 on A2.hier_id = H2.Id
    left join dbo.HIER H3 on H2.ParentId = H3.Id
    left join dbo.ABBR A3 on A3.hier_id = H3.Id
    left join dbo.HIER H4 on H3.ParentId = H4.Id
    left join dbo.ABBR A4 on A4.hier_id = H4.Id
    left join dbo.HIER H5 on H4.ParentId = H5.Id
    left join dbo.ABBR A5 on A5.hier_id = H5.Id
    where H1.id = @hier_id)
end
GO

Add columns to the table itself

For example the fullpath column, if you need, add the other 2 columns in the CTE by splitting the result of dbo.GetHierPath on ':' (left=>path, right=>abbreviations)

-- index maximum key length is 900, based on your data, 400 is enough
ALTER TABLE HIER ADD FullPath VARCHAR(400)

Maintain the columns

Because of the hierarchical nature, record X could be deleted that affects a Y descendent and Z ancestor, which is quite hard to identify in either of INSTEAD OF or AFTER triggers. So the alternative approach is based on the conditions

  • if data changes in any of the tables the view joins on, then the view needs to update itself.
  • the non-indexed view using a CTE takes 3-5 seconds to run which is way too long for my needs

We maintain the data simply by running through the entire table again, taking 3-5 seconds per update (or faster if the 5-join query works out better).

CREATE TRIGGER TG_HIER
ON HIER
AFTER INSERT, UPDATE, DELETE
AS
UPDATE HIER
SET FullPath = dbo.GetHierPath(HIER.Id)

Finally, index the new column(s) on the table itself

create index ix_hier_fullpath on HIER(FullPath)

If you intended to access the path data via the id, then it is already in the table itself without adding an additional index.

The above TSQL references these objects

Modify the table and column names to suit your schema.

CREATE TABLE dbo.HIER (Id INT Primary Key Clustered, [Name] VARCHAR(20) ,ParentId INT)
;
INSERT dbo.HIER( Id, Name, ParentId ) VALUES 
     (1, 'Europe', NULL) 
    ,(2, 'Asia',   NULL) 
    ,(3, 'Germany', 1) 
    ,(4, 'UK',      1) 
    ,(5, 'China',   2) 
    ,(6, 'India',   2) 
    ,(7, 'Scotland', 4) 
    ,(8, 'Edinburgh', 7) 
    ,(9, 'Leith', 8)
    ,(10, 'Antartica', NULL) 
; 
CREATE TABLE dbo.ABBR (id int primary key clustered, abbreviation varchar(10), hier_id int)
;
INSERT dbo.ABBR( Id, Abbreviation, hier_id ) VALUES 
     (100, 'EU', 1) 
    ,(101, 'AS', 2) 
    ,(102, 'DE', 3) 
    ,(103, 'CN', 5)
GO

EDIT - Possibly faster alternative

Given that all records are recalculated each time, there is no real need for a function that returns the FullPath for a single HIER.ID. The query in the support function can be used without the where H1.id = @hier_id filter at the end. Furthermore, the expression for FullPath can be broken into PathOnly and Abbreviation easily down the middle. Or just use the original CTE, whichever is faster.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文