我想在INIT脚本中使用Databricks Secret。
我可以使用 envvar = {{/secrets/myScope/mySecret}}}
手动设置环境变量,然后在init脚本中使用envvar。但是这样做,我将不得不在每个群集和任何新群集上手动设置该环境变量,而这些变量在我的情况下都无法使用。
理想情况下,我想通过集群策略对其进行管理,但是我看不到那里可用。
我还尝试使用一个单独的初始脚本
export envvar =“ {{{secrets/myScope/mySecret}}}”
但是,当我尝试回声时,这样做不会返回,所以我认为它是失败的。
有人知道我如何实现这一目标吗?
谢谢
垫
I would like to use a databricks secret in an init script.
I can manually set an environment variable with ENVVAR={{/secrets/myscope/mysecret}}
and then use ENVVAR in the init script. However doing it this way I would have to manually set that environment variable on each cluster, and any new ones, which won't work in my scenario.
Ideally I would like to manage it with cluster policies, but I don't see that being available there.
I've also tried to have a separate init script with
export ENVVAR="{{secrets/myscope/mysecret}}"
But doing this when I try to echo that ENVVAR it returns nothing so I assume it is failing.
Does anyone have an idea on how I can achieve this?
Thanks
Mat
发布评论
评论(1)
您通过使用初始脚本在正确的轨道上,但是您无法使用正常
导出
以来,该变量仅适用于init Script subprocess。取而代之的是,使用INIT脚本中的以下行在全球设置环境变量:
这将写入群集的
环境
文件,该文件将从群集上的任何子过程中读取。更好的方法:
甚至比在init脚本内部设置秘密的是直接在代码中使用databricks secrets api,例如:
https://docs.databricks.com/dev-tools/databricks-utils.html#dbutils-secrets
You were on the right track by using an init script, but you cannot use normal
export
since then the variable will only be available to the init script subprocess.Instead, use the following line in an init script to set an environment variable globally:
This will write to the
environment
file of the cluster, which is read from any subprocess on the cluster.Better way:
Even better than setting the secret inside an init script is to use Databricks secrets API directly in code, e.g. like this:
Read more here: https://docs.databricks.com/dev-tools/databricks-utils.html#dbutils-secrets