在Spark Scala中读取具有特殊字符的文件＆＃x27; {＆＃x27;和＆＃x27;}＆＃x27;在他们的文件名中
我想阅读具有名称的Spark Scala中的文件：月lylypurchasefile {202205} -may.txt
我正在使用以下代码：val df = spark.read.text（“ handel_special_ch/normypurchasefile {202205} -may.txt”“
但我要低于异常：
org.apache.spark.sql.AnalysisException: Path does not exist: file:/home/hdp_batch_datalake_dev/handel_special_ch/monthlyPurchaseFile{202205}-May.TXT
at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$checkAndGlobPathIfNecessary$3(DataSource.scala:792)
at org.apache.spark.util.ThreadUtils$.$anonfun$parmap$2(ThreadUtils.scala:372)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
请建议，我如何读取具有字符{code> {{，}以其名称。
I wanted to read a file in Spark Scala having name: monthlyPurchaseFile{202205}-May.TXT
I am using below code:val df = spark.read.text("handel_special_ch/monthlyPurchaseFile{202205}-May.TXT"
But I am getting below exception:
org.apache.spark.sql.AnalysisException: Path does not exist: file:/home/hdp_batch_datalake_dev/handel_special_ch/monthlyPurchaseFile{202205}-May.TXT
at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$checkAndGlobPathIfNecessary$3(DataSource.scala:792)
at org.apache.spark.util.ThreadUtils$.$anonfun$parmap$2(ThreadUtils.scala:372)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
Please suggest, how I can read that file having character {, } in its name.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
路径您将传递到spark.text方法被视为正则表达式。由于{和}是特殊字符，因此Spark试图将路径与该表达式匹配。您可以使用？字符匹配任何字符，因此应有效：The
pathyou are passing to thespark.read.textmethod is treated as a regular expression. Since{and}are special characters, Spark tries to match a path against that expression. You can use the?character to match any character, so the following should work:字符
\\用作逃生序列。因此，使用以下代码根据期望工作并解决问题：val df = spark.read.text（“ handel_special_ch/normypurchasefile \\ {202205 \\} - 五月Character
\\works as escape sequence. So, using below code works as per expectation and solves the issue:val df = spark.read.text("handel_special_ch/monthlyPurchaseFile\\{202205\\}-May.TXT"