加快 QoQ 速度还是其他方法?

发布于 2024-11-05 10:25:14 字数 440 浏览 0 评论 0原文

我正在构建一个应用程序,该应用程序执行具有多个连接的主查询。然后,整个应用程序可以使用该查询数据在全局变量中使用。查询在每次页面刷新时刷新或者获取最新的结果集;所以它只在请求的生命周期内处于相同的状态。

在此应用程序的其他部分中,我有时会对此数据运行数百个 QoQ - 通常是递归函数调用的结果。然而,虽然 QoQ 是一个很棒的功能,但它并不太快,有时页面加载在糟糕的一天可能会在 3000 - 5000 毫秒之间。只是速度不够快。

我可以采取任何类型的优化技术来使 QoQ 执行得更快,或者也许有替代方法?我读了 Ben Nadel 写的一篇关于 Duplicate() 函数的有趣文章 - 是否有使用该函数的范围,如果有,如何使用?

我很想听听你的想法。

不要担心疯狂的建议,这是一个个人项目,所以我愿意承担风险。我在与 CF8 兼容的 Railo 上运行它。

非常感谢, 迈克尔.

I am building an application that performs a master query with many joins. This query data is then available to the whole application to play around with in a global variable. The query refreshes or gets the latest result set on each page refresh; so it's only in the same state for the life of the request.

In other parts of this application, I sometimes run 100's of QoQ's on this data - usually the result of recursive function calls. However, while QoQ is a great feature, it's not too fast and sometimes page loads can be between 3000 - 5000 ms on a bad day. It's just not fast enough.

Is there any kind of optimisation techniques I can do to make QoQ perform faster or perhaps an alternative method? I read an interesting article by Ben Nadel on Duplicate() function - is there any scope for using that and if so, how?

I would love to hear your thoughts.

Don't worry about crazy suggestions, this is a personal project so I'm willing to take risks. I'm running this on Railo compatible with CF8.

Many thanks,
Michael.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

沧笙踏歌 2024-11-12 10:25:14

如果没有看到 QoQ 的代码和复杂性,很难确定最好的方法,但是您可以做的一件事是使用结构来索引 QoQ 之外的记录。使用 QoQ 的大部分开销是构建新的查询对象,并且使用结构只写方法比循环原始查询并进行比较要高效得多。

例如:

<!--- build up index --->
<cfset structindex = {} />
<cfset fields = "first,last,company" />
<cfloop list="#fields#" index="field">
    <cfset key = "field:#field#,value:#q[field][currentrow]#" />
    <!--- initialize each key (instead of using stuctkeyexists) --->
    <cfloop query="q">
        <cfset structindex[key] = "" />
    </cfloop>
    <cfloop query="q">
        <!--- update each key with list of matching row indexes --->
        <cfset structindex[key] = listappend(structindex[key], currentrow) />
    </cfloop>
</cfloop>

<!--- save structindex to global variable --->

<!--- output rows matching index --->
<cfset key = "field:company,value:stackexchange" />
<cfoutput>
    <cfloop list="#structindex[key]#" index="row">
        #q.last[row]#, #q.first[row]# (#q.company[row]#)<br />
    </cfloop>
</cfoutput>

如果这不符合您的需求,请提供一些 QoQ 语句示例以及主查询中有多少条记录。

Without seeing the code and complexity of the QoQs it is hard to say for sure the best approach, however one thing you can do is use a struct to index the records outside of a QoQ. Much of the overhead of using QoQ is building new query objects, and using a struct write only approach is much more efficient than for example looping over the original query and making comparisons.

For example:

<!--- build up index --->
<cfset structindex = {} />
<cfset fields = "first,last,company" />
<cfloop list="#fields#" index="field">
    <cfset key = "field:#field#,value:#q[field][currentrow]#" />
    <!--- initialize each key (instead of using stuctkeyexists) --->
    <cfloop query="q">
        <cfset structindex[key] = "" />
    </cfloop>
    <cfloop query="q">
        <!--- update each key with list of matching row indexes --->
        <cfset structindex[key] = listappend(structindex[key], currentrow) />
    </cfloop>
</cfloop>

<!--- save structindex to global variable --->

<!--- output rows matching index --->
<cfset key = "field:company,value:stackexchange" />
<cfoutput>
    <cfloop list="#structindex[key]#" index="row">
        #q.last[row]#, #q.first[row]# (#q.company[row]#)<br />
    </cfloop>
</cfoutput>

If this doesn't match your need provide some examples of the QoQ statements and how many records are in the main query.

你与昨日 2024-11-12 10:25:14

首先,我会查看主查询所花费的时间。如果它可以缓存一段时间并且占用了很大一部分页面加载时间,我会缓存它。

接下来,我将查看递归调用。如果它们可以迭代,那可能会加快速度。我意识到这并不总是可能的。如果这不是你最浪费时间的事情,我会感到惊讶。但是,如果不更多地了解您正在做什么,就很难帮助您优化它。

我还可能考虑在数据库服务器上编写一些递归 QoQs 存储过程,该存储过程旨在快速处理数据并有效地进行切片和切块。 CF 不是——QoQ 非常有用,但不是速度恶魔(正如您所指出的)。

最后,我会寻找简单的过滤器,而不是使用 QoQ。相反,我只是在标准 cfoutput 标记中的主查询上运行一个循环,并动态过滤。这意味着您将循环主查询一次,而不是循环主查询一次和结果查询一次。

First, I would look at the time taken by the master query. If it can be cached for some mount of time and is taking a good chunk of the pageload time, I would cache it.

Next, I would look at the recursive calls. If they can be made iterative, that would probably speed things up. I realize this is not always possible. I would be surprised if this isn't your biggest time sink. without knowing more about what you are doing, though, it's hard to help you optimize this.

I might also consider writing some of the recursive QoQs s stored procedures on the DB server, which is designed to handle data quickly and slice and dice efficiently. CF is not -- QoQs are very useful, but not speed demons (as you've noted).

Finally, I would look for straightfoward filters, and not use QoQ. Rather, I would just run a loop over the master query in a standard cfoutput tag, and filter on the fly. This means you are looping over the master query once, rather than the master query once and the result query once.

苦妄 2024-11-12 10:25:14

这里有两个主要的解决方案。首先,您可以在 CF 中对 QoQ 之外的记录执行某些操作。我已经就此发表了我的建议。另一个是在数据库中执行所有操作。我发现执行此操作的一种方法是使用子查询作为临时表。您甚至可以将 sql 语句保留在全局变量中,然后在当前使用 QoQ 但对数据库进行实际查询的相同位置引用它。听起来可能比一次访问数据库慢,然后再慢很多次,但实际上,如果索引有效的话,可能不会慢。

select *
from (
    #sqlstring#
) as tmp
where company = 'stackexchange'

实际上,我已经为具有复杂标准的系统完成了此操作,该标准既规定用户应有权访问哪些记录,又规定他们可以在这些记录中过滤哪些记录。采用这种方法意味着您始终知道内部记录的来源,而不是试图确保每个查询都正确提取。

编辑:
实际上,尽可能使用查询参数更安全(通常也更有效)。我发现这可以通过包含 sql 语句的文件来完成...

select *
from (
    <cfinclude template="master_subquery.cfm" />
) as tmp
where company = 'stackexchange'

There are two primary solutions here. First you could do something in CF with the records outside of QoQ. I posted my suggestion on this already. The other is to do everything in the db. One way I've found to do this is to use a subquery as a temp table. You can even keep the sql statement in a global variable and then reference it in the same places you are currently with the QoQ but doing a real query to the database. It may sound slower than one trip tothe DB and then many QoQ but in reality it probably isn't if indexed efficiently.

select *
from (
    #sqlstring#
) as tmp
where company = 'stackexchange'

I have actually done this for system with complex criteria for both what records a user should have access to and then also what they can filter for in those records. Going with this approach means you always know the source of the inner records instead of trying to ensure every single query is pulling correctly.

Edit:
It is actually safer (and usually more efficient) to use queryparams when ever possible. I found this can be done by including a file of the sql statement...

select *
from (
    <cfinclude template="master_subquery.cfm" />
) as tmp
where company = 'stackexchange'
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文