在Spark SQL中使用map datatype查询配置单元表时出错。但是在HiveQL中执行时工作

Question

我有下面结构的蜂房桌子

+---------------+--------------+----------------------+
| column_value  | metric_name  |         key          |
+---------------+--------------+----------------------+
| A37B          | Mean         | {0:"202006",1:"1"}  |
| ACCOUNT_ID    | Mean         | {0:"202006",1:"2"}  |
| ANB_200       | Mean         | {0:"202006",1:"3"}  |
| ANB_201       | Mean         | {0:"202006",1:"4"}  |
| AS82_RE       | Mean         | {0:"202006",1:"5"}  |
| ATTR001       | Mean         | {0:"202007",1:"2"}  |
| ATTR001_RE    | Mean         | {0:"202007",1:"3"}  |
| ATTR002       | Mean         | {0:"202007",1:"4"}  |
| ATTR002_RE    | Mean         | {0:"202007",1:"5"}  |
| ATTR003       | Mean         | {0:"202008",1:"3"}  |
| ATTR004       | Mean         | {0:"202008",1:"4"}  |
| ATTR005       | Mean         | {0:"202008",1:"5"}  |
| ATTR006       | Mean         | {0:"202009",1:"4"}  |
| ATTR006       | Mean         | {0:"202009",1:"5"}  |

select * from your_data where key[0] between  '202006' and '202009' and key NOT IN ( map(0,"202009",1,"5") );

但是当我在Spark SQL中尝试相同的查询时。我犯了错误

由于数据类型不匹配，无法解析：在org.apache.spark.sql.catalyst.analysis.package$analysiserrorat.failanalysis(package.scala:42)在org.apache.spark.sql.catalyst.analysis.checkanalysis$$$anonfun$checkanalysis$1$$anonfun$apply$3.applyrelse(checkanalysis.scalysis$115)在org.apache.spark.sql.catalyst.trees.treenode$$anone fun$transformUp$1.在org.apache.spark.sql.catalyst.trees.treenode$$$anonfun$transformUp$1.在org.apache.spark.catalyst.trees.treenode$$$anonfun$1.在org.apache.spark.catalyst.treenode$$$anonfun$transformUp$1.在org.apache.spark.scark.catalyst.treenode$$$anonfun$transformUp$1.在k.sql.catalyst.trees.treenode$$AnonFun$3.在org.apache.spark.sql.catalyst.trees.treenode应用(Treenode.scala:275)$$Anonfun$4.在org.apache.spark.sql.catalyst.trees.treenode.mapproductIterator（treenode.scala:187)在org.apache.spark.sql.catalyst.trees.mapchildre(treenode.scala:324)在org.apache.spark.sql.catalyst.trees.transformup（treenode.scala:275)在org.apache.spark.sql.catalyst.trees.treenode.transformup（l.catalyst.trees.treenode$$anonfun$3.应用(treenode.scala:275)在org.apache.spark.sql.catalyst.trees.treenode$$anonfun$4.应用(treenode.scala:326)

请救命！

Answer 1

我从我以前提出的不同问题中得到了答案。这个查询运行良好

从your_data中选择*，其中键[0]在202006和202009之间，而不是(key[0]=“202009”和key[1]=“5”）；