Scala UDF没有在雪板中运行
我已经使用 Snowpark scala 在 Snowflake 中创建并注册了临时 udf。我的代码附在下面。当我尝试运行此代码时出现错误。 我假设雪花 udf 还有其他事情要做。任何线索都会有很大帮助
session.udf.registerTemporary("lookupFromArrayUdf", (s: Array[String], schema: String, table: String, config: String, colName: String, action: String) => {
implicit val formats: DefaultFormats.type = net.liftweb.json.DefaultFormats
....code goes here....
})
我收到的错误如下
Exception in thread "main" net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error: Found a function matching HASHSENSITIVESTRINGUDF, but IMPORTS or TARGET_PATH could not be resolved.
at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowExceptionSub(SnowflakeUtil.java:127)
at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowException(SnowflakeUtil.java:67)
at net.snowflake.client.core.StmtUtil.pollForOutput(StmtUtil.java:442)
at net.snowflake.client.core.StmtUtil.execute(StmtUtil.java:345)
at net.snowflake.client.core.SFStatement.executeHelper(SFStatement.java:487)
at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:198)
at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:135)
at net.snowflake.client.core.SFStatement.describe(SFStatement.java:154)
at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.describeSqlIfNotTried(SnowflakePreparedStatementV1.java:96)
at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.getMetaData(SnowflakePreparedStatementV1.java:549)
at com.snowflake.snowpark.internal.ServerConnection.$anonfun$getResultAttributes$1(ServerConnection.scala:500)
at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:810)
at com.snowflake.snowpark.internal.ServerConnection.getResultAttributes(ServerConnection.scala:478)
at com.snowflake.snowpark.Session.getResultAttributes(Session.scala:841)
at com.snowflake.snowpark.internal.SchemaUtils$.analyzeAttributes(SchemaUtils.scala:44)
at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.attributes$lzycompute(SnowflakePlan.scala:39)
at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.attributes(SnowflakePlan.scala:38)
at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.output$lzycompute(SnowflakePlan.scala:94)
at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.output(SnowflakePlan.scala:94)
at com.snowflake.snowpark.DataFrame.output$lzycompute(DataFrame.scala:2580)
at com.snowflake.snowpark.DataFrame.output(DataFrame.scala:2580)
at com.snowflake.snowpark.DataFrame.withColumns(DataFrame.scala:1877)
at com.snowflake.snowpark.DataFrame.withColumn(DataFrame.scala:1834)
at us.company.snowpark.app.SnowparkAppUdfDriver$.$anonfun$processVariantColumn$2(SnowparkAppUdfDriver.scala:364)
at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
at scala.collection.immutable.List.foldLeft(List.scala:91)
at us.company.snowpark.app.SnowparkAppUdfDriver$.processVariantColumn(SnowparkAppUdfDriver.scala:364)
at us.company.snowpark.app.SnowparkAppUdfDriver$.$anonfun$applyHashing$6(SnowparkAppUdfDriver.scala:425)
at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:196)
at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:194)
at scala.collection.Iterator.foreach(Iterator.scala:943)
at scala.collection.Iterator.foreach$(Iterator.scala:943)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:181)
at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:199)
at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:192)
at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:108)
at us.company.snowpark.app.SnowparkAppUdfDriver$.applyHashing(SnowparkAppUdfDriver.scala:425)
at us.company.snowpark.app.SnowparkAppUdfDriver$.process(SnowparkAppUdfDriver.scala:454)
at us.company.snowpark.app.SnowparkAppUdfDriver$.main(SnowparkAppUdfDriver.scala:178)
at us.company.snowpark.app.SnowparkAppUdfDriver.main(SnowparkAppUdfDriver.scala)
I have created and registered temporary udf in snowflake using snowpark scala. My code is attached below. I am getting error when I try to run this code.
I am assuming there is something else to do with snowflake udf. Any leads would be of great help
session.udf.registerTemporary("lookupFromArrayUdf", (s: Array[String], schema: String, table: String, config: String, colName: String, action: String) => {
implicit val formats: DefaultFormats.type = net.liftweb.json.DefaultFormats
....code goes here....
})
THe error I am getting is below
Exception in thread "main" net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error: Found a function matching HASHSENSITIVESTRINGUDF, but IMPORTS or TARGET_PATH could not be resolved.
at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowExceptionSub(SnowflakeUtil.java:127)
at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowException(SnowflakeUtil.java:67)
at net.snowflake.client.core.StmtUtil.pollForOutput(StmtUtil.java:442)
at net.snowflake.client.core.StmtUtil.execute(StmtUtil.java:345)
at net.snowflake.client.core.SFStatement.executeHelper(SFStatement.java:487)
at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:198)
at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:135)
at net.snowflake.client.core.SFStatement.describe(SFStatement.java:154)
at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.describeSqlIfNotTried(SnowflakePreparedStatementV1.java:96)
at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.getMetaData(SnowflakePreparedStatementV1.java:549)
at com.snowflake.snowpark.internal.ServerConnection.$anonfun$getResultAttributes$1(ServerConnection.scala:500)
at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:810)
at com.snowflake.snowpark.internal.ServerConnection.getResultAttributes(ServerConnection.scala:478)
at com.snowflake.snowpark.Session.getResultAttributes(Session.scala:841)
at com.snowflake.snowpark.internal.SchemaUtils$.analyzeAttributes(SchemaUtils.scala:44)
at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.attributes$lzycompute(SnowflakePlan.scala:39)
at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.attributes(SnowflakePlan.scala:38)
at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.output$lzycompute(SnowflakePlan.scala:94)
at com.snowflake.snowpark.internal.analyzer.SnowflakePlan.output(SnowflakePlan.scala:94)
at com.snowflake.snowpark.DataFrame.output$lzycompute(DataFrame.scala:2580)
at com.snowflake.snowpark.DataFrame.output(DataFrame.scala:2580)
at com.snowflake.snowpark.DataFrame.withColumns(DataFrame.scala:1877)
at com.snowflake.snowpark.DataFrame.withColumn(DataFrame.scala:1834)
at us.company.snowpark.app.SnowparkAppUdfDriver$.$anonfun$processVariantColumn$2(SnowparkAppUdfDriver.scala:364)
at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
at scala.collection.immutable.List.foldLeft(List.scala:91)
at us.company.snowpark.app.SnowparkAppUdfDriver$.processVariantColumn(SnowparkAppUdfDriver.scala:364)
at us.company.snowpark.app.SnowparkAppUdfDriver$.$anonfun$applyHashing$6(SnowparkAppUdfDriver.scala:425)
at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:196)
at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:194)
at scala.collection.Iterator.foreach(Iterator.scala:943)
at scala.collection.Iterator.foreach$(Iterator.scala:943)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:181)
at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:199)
at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:192)
at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:108)
at us.company.snowpark.app.SnowparkAppUdfDriver$.applyHashing(SnowparkAppUdfDriver.scala:425)
at us.company.snowpark.app.SnowparkAppUdfDriver$.process(SnowparkAppUdfDriver.scala:454)
at us.company.snowpark.app.SnowparkAppUdfDriver$.main(SnowparkAppUdfDriver.scala:178)
at us.company.snowpark.app.SnowparkAppUdfDriver.main(SnowparkAppUdfDriver.scala)
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
问题:我只是将依赖项罐放在舞台上,但是应用程序罐还需要在雪花舞台
分辨率中存在:当我将整个应用程序罐上传到舞台时,问题就解决了。
Problem: I just kept the dependencies jar in the stage but the application jar is also needed to be present in Snowflake Stage
Resolution: The issue got resolved when I uploaded the entire application jar to the stage.