使用 createOrReplaceTempView 替换未按预期工作的临时视图

Question

我有一个与此类似的数据集

{"name": "Michael", "age": "30", "producta1": "blah1", "producta3": "blah2"}
{"name": "Michael", "age": "31", "producta1": "blah3", "producta3": "blah4"}
{"name": "Michael", "age": "30", "producta1": "blah5", "producta3": "blah6"}
{"name": "Andy", "age": "28", "producta1": "blah5", "producta3": "blah6"}
{"name": "Andy", "age": "28", "producta1": "blah5", "producta3": "blah6"}
{"name": "Andy", "age": "28", "producta1": "blah5", "producta3": "blah6"}
{"name": "Justin", "age": "12", "producta1": "blah5", "producta3": "blah6"}
{"name": "Justin", "age": "12", "producta1": "blah5", "producta3": "blah6"}

我的 Spark 代码是

SparkSession sc = SparkSession.builder().appName("example app").config("spark.master","local").getOrCreate();

Dataset<Row> df = sc.read().json("/Users/g.bhageshpur/Downloads/spark-master/examples/src/main/examples/src/main/resources/people.json");

df.createOrReplaceTempView("people");

Dataset<Row> dfpeople = sc.sql("select * from people where name='Michael'");

dfpeople.createOrReplaceTempView("people");

我尝试通过调用 createOrReplaceTempView 来替换人员视图

，但出现以下错误

Recursive view `people` detected (cycle: `people` -> `people`)

如何替换 Spark 中的视图？

Answer 1

所以我通过以下代码得到了上述问题的解决方案

Dataset<Row> df = sc.read().json("/Users/g.bhageshpur/Downloads/spark-master/examples/src/main/examples/src/main/resources/people.json");

df.createOrReplaceTempView("people");

Dataset<Row> dfpeople = sc.sql("select * from people where name='Michael'");

sc.sqlContext().dropTempTable("people");

dfpeople.createOrReplaceTempView("people");