Scala UDF没有在雪板中运行

发布于 2025-01-19 10:03:24 字数 4499 浏览 2 评论 0原文

我已经使用 Snowpark scala 在 Snowflake 中创建并注册了临时 udf。我的代码附在下面。当我尝试运行此代码时出现错误。 我假设雪花 udf 还有其他事情要做。任何线索都会有很大帮助

session.udf.registerTemporary("lookupFromArrayUdf", (s: Array[String], schema: String, table: String, config: String, colName: String, action: String) => {
      implicit val formats: DefaultFormats.type = net.liftweb.json.DefaultFormats
      ....code goes here....
    })

我收到的错误如下

Exception in thread "main" net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error: Found a function matching HASHSENSITIVESTRINGUDF, but IMPORTS or TARGET_PATH could not be resolved.
        at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowExceptionSub(SnowflakeUtil.java:127)
        at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowException(SnowflakeUtil.java:67)
        at net.snowflake.client.core.StmtUtil.pollForOutput(StmtUtil.java:442)
        at net.snowflake.client.core.StmtUtil.execute(StmtUtil.java:345)
        at net.snowflake.client.core.SFStatement.executeHelper(SFStatement.java:487)
        at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:198)
        at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:135)
        at net.snowflake.client.core.SFStatement.describe(SFStatement.java:154)
        at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.describeSqlIfNotTried(SnowflakePreparedStatementV1.java:96)
        at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.getMetaData(SnowflakePreparedStatementV1.java:549)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$getResultAttributes$1(ServerConnection.scala:500)
        at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:810)
        at com.snowflake.snowpark.internal.ServerConnection.getResultAttributes(ServerConnection.scala:478)
        at com.snowflake.snowpark.Session.getResultAttributes(Session.scala:841)
        at com.snowflake.snowpark.internal.SchemaUtils$.analyzeAttributes(SchemaUtils.scala:44)
        at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.attributes$lzycompute(SnowflakePlan.scala:39)
        at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.attributes(SnowflakePlan.scala:38)
        at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.output$lzycompute(SnowflakePlan.scala:94)
        at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.output(SnowflakePlan.scala:94)
        at com.snowflake.snowpark.DataFrame.output$lzycompute(DataFrame.scala:2580)
        at com.snowflake.snowpark.DataFrame.output(DataFrame.scala:2580)
        at com.snowflake.snowpark.DataFrame.withColumns(DataFrame.scala:1877)
        at com.snowflake.snowpark.DataFrame.withColumn(DataFrame.scala:1834)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.$anonfun$processVariantColumn$2(SnowparkAppUdfDriver.scala:364)
        at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
        at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
        at scala.collection.immutable.List.foldLeft(List.scala:91)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.processVariantColumn(SnowparkAppUdfDriver.scala:364)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.$anonfun$applyHashing$6(SnowparkAppUdfDriver.scala:425)
        at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:196)
        at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:194)
        at scala.collection.Iterator.foreach(Iterator.scala:943)
        at scala.collection.Iterator.foreach$(Iterator.scala:943)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
        at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:181)
        at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:199)
        at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:192)
        at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:108)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.applyHashing(SnowparkAppUdfDriver.scala:425)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.process(SnowparkAppUdfDriver.scala:454)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.main(SnowparkAppUdfDriver.scala:178)
        at us.company.snowpark.app.SnowparkAppUdfDriver.main(SnowparkAppUdfDriver.scala)

I have created and registered temporary udf in snowflake using snowpark scala. My code is attached below. I am getting error when I try to run this code.
I am assuming there is something else to do with snowflake udf. Any leads would be of great help

session.udf.registerTemporary("lookupFromArrayUdf", (s: Array[String], schema: String, table: String, config: String, colName: String, action: String) => {
      implicit val formats: DefaultFormats.type = net.liftweb.json.DefaultFormats
      ....code goes here....
    })

THe error I am getting is below

Exception in thread "main" net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error: Found a function matching HASHSENSITIVESTRINGUDF, but IMPORTS or TARGET_PATH could not be resolved.
        at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowExceptionSub(SnowflakeUtil.java:127)
        at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowException(SnowflakeUtil.java:67)
        at net.snowflake.client.core.StmtUtil.pollForOutput(StmtUtil.java:442)
        at net.snowflake.client.core.StmtUtil.execute(StmtUtil.java:345)
        at net.snowflake.client.core.SFStatement.executeHelper(SFStatement.java:487)
        at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:198)
        at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:135)
        at net.snowflake.client.core.SFStatement.describe(SFStatement.java:154)
        at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.describeSqlIfNotTried(SnowflakePreparedStatementV1.java:96)
        at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.getMetaData(SnowflakePreparedStatementV1.java:549)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$getResultAttributes$1(ServerConnection.scala:500)
        at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:810)
        at com.snowflake.snowpark.internal.ServerConnection.getResultAttributes(ServerConnection.scala:478)
        at com.snowflake.snowpark.Session.getResultAttributes(Session.scala:841)
        at com.snowflake.snowpark.internal.SchemaUtils$.analyzeAttributes(SchemaUtils.scala:44)
        at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.attributes$lzycompute(SnowflakePlan.scala:39)
        at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.attributes(SnowflakePlan.scala:38)
        at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.output$lzycompute(SnowflakePlan.scala:94)
        at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.output(SnowflakePlan.scala:94)
        at com.snowflake.snowpark.DataFrame.output$lzycompute(DataFrame.scala:2580)
        at com.snowflake.snowpark.DataFrame.output(DataFrame.scala:2580)
        at com.snowflake.snowpark.DataFrame.withColumns(DataFrame.scala:1877)
        at com.snowflake.snowpark.DataFrame.withColumn(DataFrame.scala:1834)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.$anonfun$processVariantColumn$2(SnowparkAppUdfDriver.scala:364)
        at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
        at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
        at scala.collection.immutable.List.foldLeft(List.scala:91)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.processVariantColumn(SnowparkAppUdfDriver.scala:364)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.$anonfun$applyHashing$6(SnowparkAppUdfDriver.scala:425)
        at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:196)
        at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:194)
        at scala.collection.Iterator.foreach(Iterator.scala:943)
        at scala.collection.Iterator.foreach$(Iterator.scala:943)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
        at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:181)
        at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:199)
        at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:192)
        at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:108)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.applyHashing(SnowparkAppUdfDriver.scala:425)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.process(SnowparkAppUdfDriver.scala:454)
        at us.company.snowpark.app.SnowparkAppUdfDriver$.main(SnowparkAppUdfDriver.scala:178)
        at us.company.snowpark.app.SnowparkAppUdfDriver.main(SnowparkAppUdfDriver.scala)

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

爱情眠于流年 2025-01-26 10:03:24

问题:我只是将依赖项罐放在舞台上,但是应用程序罐还需要在雪花舞台

分辨率中存在:当我将整个应用程序罐上传到舞台时,问题就解决了。

Problem: I just kept the dependencies jar in the stage but the application jar is also needed to be present in Snowflake Stage

Resolution: The issue got resolved when I uploaded the entire application jar to the stage.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文