向“查询”提供 SQL 字符串时,Power Query Sql.Database 速度很慢与 GUI 生成的 M 代码相比的选项
我发现将 Sql.Database
与 [Query=
选项一起使用会比使用 Sql.Database 慢得多
没有指定查询,而是深入到结果表中。
例如,如果我通过 Power Query GUI 创建 SQL 查询,则可以生成以下形式的查询:
查询 1:
let
Source = Sql.Database("SomeServer", "SomeDB"),
SomeTable = Source{[Schema="SomeSchema", Item="SomeTable"})[Data],
SelectedCols = Table.SelectColumns(SomeTable, {"a", "b", "c"}),
FilteredRows = Table.SelectRows(SelectedCols, each [a] = 1 and [b] = 2)
Result = FilteredRows
in
Result
但是,如果我选择“查看本机查询”选项,并将该 SQL 代码直接粘贴到以下查询中,这将导致结果慢得多:
查询 2:
let
// this uses the result of the native query generated by Query 1
sql_string = "
select [_].[a], [_].[b], [_].[c]
from (select [a], [b], [c] from [SomeSchema].[SomeTable] as [$Table]) as [_]
where [_].[a] = 1 and [_].[b] = 2
",
run_query = (sql_string as text) as table => Sql.Database("SomeServer", "SomeDB", [Query=sql_string])
Result = run_query(sql_string)
in
Result
在我的例子中,一个更复杂的查询生成超过 1M 行,我发现第一个查询在 1-2 秒内执行,而第二个查询则需要近 5 分钟。
如果查询 1 应该运行本机查询,而查询 2 只是直接启动相同的查询,那么这怎么会有这么大的不同呢?请注意,数据库位于 SQL Server 中。
I am finding that using Sql.Database
with the [Query=<query-string>]
option is leading to much slower query times than using Sql.Database
without a query specified and instead followed by drilling down into the resulting table.
For example, if I create an SQL query through the Power Query GUI, I can produce a query of this form:
Query 1:
let
Source = Sql.Database("SomeServer", "SomeDB"),
SomeTable = Source{[Schema="SomeSchema", Item="SomeTable"})[Data],
SelectedCols = Table.SelectColumns(SomeTable, {"a", "b", "c"}),
FilteredRows = Table.SelectRows(SelectedCols, each [a] = 1 and [b] = 2)
Result = FilteredRows
in
Result
However, if I select the "View Native Query" option, and paste that SQL code directly into the following query, this will result in a much slower result:
Query 2:
let
// this uses the result of the native query generated by Query 1
sql_string = "
select [_].[a], [_].[b], [_].[c]
from (select [a], [b], [c] from [SomeSchema].[SomeTable] as [$Table]) as [_]
where [_].[a] = 1 and [_].[b] = 2
",
run_query = (sql_string as text) as table => Sql.Database("SomeServer", "SomeDB", [Query=sql_string])
Result = run_query(sql_string)
in
Result
In my case, with a more complex query producing over 1M rows, I am finding that the first query performs in 1-2 seconds, while the second takes nearly 5 minutes.
How can this be so much different if Query 1 is supposedly running a native query, and Query 2 is just initiating that same query directly? Note the database is in SQL Server.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
如果将本机查询放在正确的位置,它将运行得很快。在第一步(源代码)中,将本机查询放入 SQL 语句框中,如下所示:
我通常不会执行 GUI 生成的本机查询,但它是一个很好的指示器。如果最后一步提供本机查询,则意味着该查询将完全在服务器上执行。如果最后一步中“native query”是灰色的,则表示该查询将在PC上执行。这意味着将数据传输到 PC 进行处理,并且在处理数百万行数据时速度可能会很慢。
我有时在 SQL Server Management Studio (SSMS) 中开发查询,然后将生成的查询粘贴到 Excel SQL Server 数据库屏幕的 SQL 语句框中。我可以按原样使用它,或者在 Excel GUI 中进一步修改查询。
If you put the native query in the correct location, it will run fast. In the first step (Source) put your native query in the SQL Statement box like this:
I don't normally execute the native query generated by the GUI, but it is a good indicator. If the last step provides a native query, it means the query will execute entirely on the server. If "native query" is greyed out in the last step, it means the query will execute on the PC. This means bringing the data to the PC for processing and can be slow when you deal with millions of rows of data.
I sometimes develop queries in SQL Server Management Studio (SSMS) then paste the resultant query in the SQL Statement box in Excel SQL Server database screen. I can use it as is or touch up the query further in Excel GUI.