Databricks Init脚本无法安装软件包,但报告“成功”不管?

发布于 2025-01-27 12:29:40 字数 470 浏览 5 评论 0原文

我一直在尝试在Databricks上设置“初始化脚本”,因此我可以安装所有Python库并控制环境。

昨天使用下图所示的初始脚本尝试了:

dbutils.fs.put("/DA/Temp/ClusterTest/python_requirements_test2.sh","""
#!/bin/bash
pip install pyodbc==4.0.32
pip install zeep==4.1.0
""", True)

这是成功的!

在试用了其他一些Techinques并回到此方法之后,Init脚本不再安装任何图书馆。

但是,

以下是有关INIT脚本的“事件日志”注释,显示“状态:成功”,即使实际上没有库:

I have been attempting to setup 'init scripts' on databricks, so I can install all of my python libraries and keep the environment controlled.

Tried yesterday using the init script pictured below:

dbutils.fs.put("/DA/Temp/ClusterTest/python_requirements_test2.sh","""
#!/bin/bash
pip install pyodbc==4.0.32
pip install zeep==4.1.0
""", True)

This was successful!

But after trialling some other techinques and coming back to this method, the init script is no longer installing any of the libraries..

Can anyone shed some light on this?

Below is the 'Event Log' note about the init script, showing "Status: SUCCEEDED" even though no libraries are actually installed:
"Status: SUCCEEDED" even though no libraries are actually installed

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

暗地喜欢 2025-02-03 12:29:40

与Databricks支持团队进行了对话后,这是其簇UI的预期功能,即 显示通过UI中安装的软件包。但是,这些软件包已安装,但这并不表示安装了这些软件包。

我已要求支持团队对此进行更新他们的文档,因为目前尚不清楚这是一种预期的行为。

After a conversation with Databricks support team, it is an intended functionality of their Clusters UI to NOT show packages installed via init script in the UI. These packages are installed however and this does not indicate that the packages arent installed.

I have asked the support team to update their documentation on this, as it was not clear that this was an intended behaviour.

貪欢 2025-02-03 12:29:40

当您在群集上安装库时,笔记本已经附加了
到该集群将不会立即看到新库。你必须
首先分离,然后将笔记本重新安排到集群中。

查看集群上安装的库

  • 单击侧边栏中的计算图标计算。
  • 单击群集名称。
  • 单击“库”选项卡。对于每个库,标签显示名称和版本,类型,安装状态以及(如果已上传)源文件。

When you install a library on a cluster, a notebook already attached
to that cluster will not immediately see the new library. You must
first detach and then reattach the notebook to the cluster.

To View the libraries installed on a cluster:

  • Click compute icon Compute in the sidebar.
  • Click the cluster name.
  • Click the Libraries tab. For each library, the tab displays the name and version, type, install status, and, if uploaded, the source file.
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文