安装ADLS GEN2存储 - 文件必须为dbfs或s3n: /

发布于 01-24 14:13 字数 874 浏览 2 评论 0原文

我想使用此代码将ADLS GEN2存储安装在数据链串中:

%python

configs = {
  "fs.azure.account.key.<storage-account-name>.dfs.core.windows.net": <storage-account-key>,
  "spark.hadoop.fs.azure.account.key.<storage-account-name>.dfs.core.windows.net": <storage-account-key>
}

dbutils.fs.mount(
  source = "abfss://<container-name>@<storage-account-name>>.dfs.core.windows.net/",
  mount_point = "/mnt/aaa",
  extra_configs = configs)

但是我得到:

”在此处输入图像描述”

“在此处输入图像描述”

有什么想法是根本原因吗?

I want to mount ADLS Gen2 Storage in Databricks with this code:

%python

configs = {
  "fs.azure.account.key.<storage-account-name>.dfs.core.windows.net": <storage-account-key>,
  "spark.hadoop.fs.azure.account.key.<storage-account-name>.dfs.core.windows.net": <storage-account-key>
}

dbutils.fs.mount(
  source = "abfss://<container-name>@<storage-account-name>>.dfs.core.windows.net/",
  mount_point = "/mnt/aaa",
  extra_configs = configs)

But I get:

enter image description here

enter image description here

Any idea what can be root cause?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

冬天的雪花 2025-01-31 14:13:52

databrick中的安装adls gen2存储

您可以在Databrick中安装并实现两种情况。

方案1:

直接从adls gen2 blob存储&lt; storage-account-access key&gt; extair_configs中,从而直接从adls gen2 blob存储中访问访问键。创建安装座。

语法

spark.conf.set("fs.azure.account.key.<storage-account-name>.dfs.core.windows.net", dbutils.secrets.get(scope="<Scope-Name>",key="Key_Value"))

dbutils.fs.mount(
    source = "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/",
    mount_point = "/mnt/io243",
    extra_configs = {"fs.azure.account.key.<storage-account-name>.blob.core.windows.net":"<storage-account-Access key>"})

​“ nofollow noreferrer”> “

方案2:

转到 访问密钥 复制键 - &gt; 创建Secret ,用于 Azure键Vault 中的访问密钥。

创建一个 azure键库 秘密范围> - &gt; 创建范围 - &gt;输入 DNS名称
(对于 示例 https://databrickskv.vault.azure.azure.net/)和 资源ID - &gt;转到 Azure Portal 中的azure键值 的属性选项卡DNS
资源ID

dbutils.fs.mount(
            source = "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/",
            mount_point = "/mnt/io243",
            extra_configs = {"fs.azure.account.key.<storage-account-name>.blob.core.windows.net":"dbutils.secrets.get(scope = "databricks-secret-scope", key = "blob-container-key")}

“

参考:

https://bigdataprogrammers.com/create-mount-point-point-in-azure-databricks/

https://learn.microsoft.com/en-us/azure/databricks/data/databricks/databricks-file-system

//www.youtube.com/watch?v=yengrbxhmcc“ rel =“ nofollow noreferrer”> https://www.youtube.com/watch?v=yengrbxhmcc

Mount ADLS Gen2 Storage in Databrick

There are two scenarios you can Mount and implement ADLS Gen2 Storage in Databrick.

Scenario 1:

Directly take the Access key from ADLS Gen2 Blob Storage past in <storage-account-Access key> of extra_configs as shown in the create mount.

Syntax

spark.conf.set("fs.azure.account.key.<storage-account-name>.dfs.core.windows.net", dbutils.secrets.get(scope="<Scope-Name>",key="Key_Value"))

Ref1

Create Mount

dbutils.fs.mount(
    source = "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/",
    mount_point = "/mnt/io243",
    extra_configs = {"fs.azure.account.key.<storage-account-name>.blob.core.windows.net":"<storage-account-Access key>"})

Ref2

Scenario 2:

Go to the Access Keys and copy the key -> Create Secret for Access Key in Azure Key Vault .

Ref3

Create an Azure Key Vault and Secret Scope -> Create Scope -> Enter the DNS Name
(for Example, https://databrickskv.vault.azure.net/) and Resource ID -> Go to Properties tab of an Azure Key Vault in your Azure portal you get both DNS and Resource ID.

Ref4

dbutils.fs.mount(
            source = "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/",
            mount_point = "/mnt/io243",
            extra_configs = {"fs.azure.account.key.<storage-account-name>.blob.core.windows.net":"dbutils.secrets.get(scope = "databricks-secret-scope", key = "blob-container-key")}

Ref5

Reference:

https://bigdataprogrammers.com/create-mount-point-in-azure-databricks/

https://learn.microsoft.com/en-us/azure/databricks/data/databricks-file-system

https://www.youtube.com/watch?v=yeNgrBxHmCc

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文