快照由于查询执行期间超过“资源”而失败。
在我的DBT管道结束时,将数据转换为BigQuery数据库中,我已经安排了一些基本快照。此快照使我能够为每天的销售预测来想象一个日历表。这是非常基本的,并不复杂。
我以前从未有任何其他快照的问题,但是对于那个我有以下错误:
"Database Error in snapshot sales_forecasts_snapshot (snapshots/sales_forecasts_snapshot.sql)
07:55:51 Resources exceeded during query execution: Not enough resources for query planning - too many subqueries or query is too complex."
这是代码:
'''{% snapshot sales_forecasts_snapshot %}
{{
config(
target_database='baobab-304613',
target_schema='snapshots',
unique_key='calendar_day',
strategy='timestamp',
updated_at='updated_at',
)
}}
select
calendar_day AS calendar_day
,CAST(total_forecast_sales_paying_members AS NUMERIC) AS total_forecast_sales_paying_members
,CAST(forecast_nb_paying_members AS NUMERIC) AS forecast_nb_paying_members
,CAST(forecast_sales_per_paying_member AS NUMERIC) AS forecast_sales_per_paying_member
,CAST(total_forecast_sales_free_members AS NUMERIC) AS total_forecast_sales_free_members
,CAST(forecast_nb_free_members AS NUMERIC) AS forecast_nb_free_members
,CAST(forecast_sales_per_free_member AS NUMERIC) AS forecast_sales_per_free_member
,CAST(total_forecast_sales_free_trialer AS NUMERIC) AS total_forecast_sales_free_trialer
,CAST(forecast_nb_free_trialers AS NUMERIC) AS forecast_nb_free_trialers
,CAST(forecast_sales_per_free_trialer AS NUMERIC) AS forecast_sales_per_free_trialer
,CAST(total_forecast_sales_active_customer_90_days AS NUMERIC) AS total_forecast_sales_active_customer_90_days
,CAST(forecast_sales_per_active_customer_90_days AS NUMERIC) AS forecast_sales_per_active_customer_90_days
,CAST(forecast_nb_active_customers_90_days AS NUMERIC) AS forecast_nb_active_customers_90_days
,updated_at AS updated_at --this date is the current_date (the date of the day --> the idea is to capture every day the new forecasts)
from {{ ref('sales_forecasts') }}
{% endsnapshot %}'''
此快照是基于在没有任何问题的情况下运行的视图。 该快照是在BigQuery数据库中使用DBT执行的。当我创建快照时,它第一次运行时运行良好,但在第二次启动时失败。 令我感到惊讶的是,即使我的管道由于此功能而失败,它似乎仍然运行,因为我在数据库中有新的数据。
At the end of my dbt pipeline that transform my data in a bigquery database I have scheduled some basic snapshots. This snap enables me to picture a calendar table with sales forecast for each day. It's quite basic and not complexe.
Never had any issue with all my other snap before but with that one I have the following error :
"Database Error in snapshot sales_forecasts_snapshot (snapshots/sales_forecasts_snapshot.sql)
07:55:51 Resources exceeded during query execution: Not enough resources for query planning - too many subqueries or query is too complex."
Here is the code :
'''{% snapshot sales_forecasts_snapshot %}
{{
config(
target_database='baobab-304613',
target_schema='snapshots',
unique_key='calendar_day',
strategy='timestamp',
updated_at='updated_at',
)
}}
select
calendar_day AS calendar_day
,CAST(total_forecast_sales_paying_members AS NUMERIC) AS total_forecast_sales_paying_members
,CAST(forecast_nb_paying_members AS NUMERIC) AS forecast_nb_paying_members
,CAST(forecast_sales_per_paying_member AS NUMERIC) AS forecast_sales_per_paying_member
,CAST(total_forecast_sales_free_members AS NUMERIC) AS total_forecast_sales_free_members
,CAST(forecast_nb_free_members AS NUMERIC) AS forecast_nb_free_members
,CAST(forecast_sales_per_free_member AS NUMERIC) AS forecast_sales_per_free_member
,CAST(total_forecast_sales_free_trialer AS NUMERIC) AS total_forecast_sales_free_trialer
,CAST(forecast_nb_free_trialers AS NUMERIC) AS forecast_nb_free_trialers
,CAST(forecast_sales_per_free_trialer AS NUMERIC) AS forecast_sales_per_free_trialer
,CAST(total_forecast_sales_active_customer_90_days AS NUMERIC) AS total_forecast_sales_active_customer_90_days
,CAST(forecast_sales_per_active_customer_90_days AS NUMERIC) AS forecast_sales_per_active_customer_90_days
,CAST(forecast_nb_active_customers_90_days AS NUMERIC) AS forecast_nb_active_customers_90_days
,updated_at AS updated_at --this date is the current_date (the date of the day --> the idea is to capture every day the new forecasts)
from {{ ref('sales_forecasts') }}
{% endsnapshot %}'''
This snapshot is based on a view which runs without any issue.
This snapshot is performed with dbt in a bigquery database. When I create the snapshot, it runs perfectly well the first time it runs but then fail at the second launch.
What surprises me is also that even if my pipeline fails because of that, it seems like it still run as I have fresh data in my database.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论