无法将DBT Cloud或DBT Core连接到Databricks

发布于 2025-01-29 23:33:04 字数 2369 浏览 3 评论 0 原文

我在将DBT云和DBT Core连接到数据映的问题上有问题,

我读了这4个链接,但仍然无法连接

https://docs.databricks.com/integrations/prep/dbt-cloud.html#connect-to-to-connect-to-dbt-dbt-cloud&dbt-cloud&amp.amp;language-cluster

https://docs.databricks.coms.coms.coms.com/integrations/integrations/intregrations/IntIntegrations/prep/prep/dbt。 html

https://docs.getdbt.com/reference/wareference/warehousehousehousehousehousehousehousehousehousehousehousehousehousehousehousehousehousehousehousehousehousehousehousehousehouse -profiles/databricks-profile

httpps://github.com/databrick.com/databricks/databricks/databricks/datbt-databricks

在dbt-cloud上:

当我在项目创建步骤中测试连接时,它通过了测试。但是,当我使用连接创建作业并运行时,它会返回此消息:“无法在Spark中设置数据库!”

编辑:问题曾经解决过,但又回来了。

原始修复: DBT核连接问题已解决。该问题是由MacOS上的Python证书问题引起的。请参阅此链接的解决方案

在DBT核上: 这是我在文档上设置propiles.yaml文件库的方式:

    databrick_dbt_lakehouse:
      outputs:
        dev:
          host: adb-755xxxxxxx7.7.azuredatabricks.net
          http_path: /sql/protocolv1/o/755xxxx7/0517-xxxxxx-xxxxxx
          schema: default
          threads: 1
          token: dapi<my token>
          type: databricks
      target: dev

注意:

  1. 对于http_path,我在sql/...我之前尝试过有或没有backslash(/)的http_path ...
  2. 我假设该架构表示数据库名称。我已经尝试了2,但是它们都没有作品,
  3. 我将使用Pipenv与Python版本3.8.8一起

运行 dbt debug 我收到了此消息:

check failed:
   dbt was unable to connect to the specified database.
   The database returned the following error:

Runtime Error
   Database Error
   failed to connect

请提供帮助,谢谢,谢谢

I am having issue connecting my dbt cloud and dbt core to databricks

I have read these 4 links, but still can not connect

https://docs.databricks.com/integrations/prep/dbt-cloud.html#connect-to-dbt-cloud&language-Cluster

https://docs.databricks.com/integrations/prep/dbt.html

https://docs.getdbt.com/reference/warehouse-profiles/databricks-profile

https://github.com/databricks/dbt-databricks

on dbt-cloud:

When I test the connection during the project creation step, it passed the test. however when I use the connection to create a job and run, it returns this message:"Cannot set database in spark!"

Edit: the issue once has been fixed but it comes back again.

original fix:
The dbt-core connection issue has been fixed. The issue is caused by the python certificate issue on MacOS. please refer to this link for the solution

on dbt-core:
this how I set up my profiles.yaml file base on the documentations:

    databrick_dbt_lakehouse:
      outputs:
        dev:
          host: adb-755xxxxxxx7.7.azuredatabricks.net
          http_path: /sql/protocolv1/o/755xxxx7/0517-xxxxxx-xxxxxx
          schema: default
          threads: 1
          token: dapi<my token>
          type: databricks
      target: dev

Note:

  1. for http_path i have tried both with or without backslash (/) before sql/...
  2. I assume the schema means database name. I have tried 2, but none of them works
  3. I use pipenv with python version 3.8.8

when I run dbt debug I got this message:

check failed:
   dbt was unable to connect to the specified database.
   The database returned the following error:

Runtime Error
   Database Error
   failed to connect

Please help, thanks

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

无语# 2025-02-05 23:33:04

您的 http_path 似乎是错误的。这是我的 profiles.yml 的摘录,

databricks_dbt_demo:
  outputs:
    dev:
      host: adb-redacted.azuredatabricks.net

      http_path: /sql/1.0/endpoints/redacted  # for SQL Endpoint
      # http_path: /sql/protocolv1/o/{ORG-ID}/{CLUSTER-ID}  # for All-Purpose Cluster
      schema: your_database_here
      token: your_personal_access_token_here
      type: databricks
  target: dev

您是使用群集(例如通用目的群集)还是SQL端点?

Your http_path seems wrong. Here is an extract from my profiles.yml

databricks_dbt_demo:
  outputs:
    dev:
      host: adb-redacted.azuredatabricks.net

      http_path: /sql/1.0/endpoints/redacted  # for SQL Endpoint
      # http_path: /sql/protocolv1/o/{ORG-ID}/{CLUSTER-ID}  # for All-Purpose Cluster
      schema: your_database_here
      token: your_personal_access_token_here
      type: databricks
  target: dev

Are you using a cluster (e.g. All Purpose Cluster) or a SQL Endpoint?

趴在窗边数星星i 2025-02-05 23:33:04

编辑:这个问题又回来了。 后的第三天写了此评论。

当我运行 dbt Run dbt快照返回此错误消息时,

Encountered an error:
Runtime Error
   Database Error
      failed to connect

在原始修复

我 MacOS上的Python证书发行。请参阅此链接的解决方案

DBT云问题是由YAML文件中错误的模式名称引起的

Edit: This issue is came back again. I am writing this comment on the 3rd day after the original fix

When I run dbt run or dbt snapshot the returns this error message:

Encountered an error:
Runtime Error
   Database Error
      failed to connect

original fix:

The issue is caused by the python certificate issue on MacOS. please refer to this link for the solution.

The dbt cloud issue is caused by the incorrect schema name in the yaml file

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文