使用createOrReplaceTempView替换未按预期工作的临时视图

Question

我有一个类似的数据集

{"name": "Michael", "age": "30", "producta1": "blah1", "producta3": "blah2"}
{"name": "Michael", "age": "31", "producta1": "blah3", "producta3": "blah4"}
{"name": "Michael", "age": "30", "producta1": "blah5", "producta3": "blah6"}
{"name": "Andy", "age": "28", "producta1": "blah5", "producta3": "blah6"}
{"name": "Andy", "age": "28", "producta1": "blah5", "producta3": "blah6"}
{"name": "Andy", "age": "28", "producta1": "blah5", "producta3": "blah6"}
{"name": "Justin", "age": "12", "producta1": "blah5", "producta3": "blah6"}
{"name": "Justin", "age": "12", "producta1": "blah5", "producta3": "blah6"}

我的星火密码是

SparkSession sc = SparkSession.builder().appName("example app").config("spark.master","local").getOrCreate();

Dataset<Row> df = sc.read().json("/Users/g.bhageshpur/Downloads/spark-master/examples/src/main/examples/src/main/resources/people.json");

df.createOrReplaceTempView("people");

Dataset<Row> dfpeople = sc.sql("select * from people where name='Michael'");

dfpeople.createOrReplaceTempView("people");

我试图通过调用createOrReplaceTempView来替换people视图

但我得到如下错误

Recursive view `people` detected (cycle: `people` -> `people`)

如何替换spark中的视图？

Answer 1

所以我通过下面的代码得到了上述问题的解决方案

Dataset<Row> df = sc.read().json("/Users/g.bhageshpur/Downloads/spark-master/examples/src/main/examples/src/main/resources/policy.json");

df.createOrReplaceTempView("policy");

Dataset<Row> dfpolicy = sc.sql("select * from policy where name='Full'");

sc.sqlContext().dropTempTable("policy");

dfpolicy.createOrReplaceTempView("policy");