Apache Flink：如何在翻滚时间窗口中使用DISTINCT？

Question

我有这样一个流：

tableEnv.registerDataStream("userVisitPage", stream, "_time.rowtime, uri,userId");

然后我查询这个表:

final String sql =
       "SELECT tumble_start(_time, interval '10' second) as timestart, " +
       "  count(distinct userId) as uv, " +
       "  uri as uri, " +
       "  count(1) as pv " +
       "FROM userVisitPage " +
       "GROUP BY tumble(_time, interval '10' second), uri";

final Table table = tableEnv.sqlQuery(sql);

但是，查询会引发异常：

org.apache.flink.table.codegen.CodeGenException: Unsupported call: TUMBLE 
If you think this function should be supported, you can create an issue and start a discussion for it.
    at org.apache.flink.table.codegen.CodeGenerator$$anonfun$visitCall$3.apply(CodeGenerator.scala:1006)
    at org.apache.flink.table.codegen.CodeGenerator$$anonfun$visitCall$3.apply(CodeGenerator.scala:1006)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.flink.table.codegen.CodeGenerator.visitCall(CodeGenerator.scala:1006)
    at org.apache.flink.table.codegen.CodeGenerator.visitCall(CodeGenerator.scala:67)
    at org.apache.calcite.rex.RexCall.accept(RexCall.java:107)
    at org.apache.flink.table.codegen.CodeGenerator.generateExpression(CodeGenerator.scala:234)
    at org.apache.flink.table.codegen.CodeGenerator$$anonfun$7.apply(CodeGenerator.scala:321)
    at org.apache.flink.table.codegen.CodeGenerator$$anonfun$7.apply(CodeGenerator.scala:321)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
    at scala.collection.AbstractTraversable.map(Traversable.scala:104)
    at org.apache.flink.table.codegen.CodeGenerator.generateResultExpression(CodeGenerator.scala:321)
    at org.apache.flink.table.plan.nodes.CommonCalc$class.generateFunction(CommonCalc.scala:44)
    at org.apache.flink.table.plan.nodes.datastream.DataStreamCalc.generateFunction(DataStreamCalc.scala:43)
    at org.apache.flink.table.plan.nodes.datastream.DataStreamCalc.translateToPlan(DataStreamCalc.scala:116)
    at org.apache.flink.table.plan.nodes.datastream.DataStreamGroupAggregate.translateToPlan(DataStreamGroupAggregate.scala:113)
    at org.apache.flink.table.plan.nodes.datastream.DataStreamGroupAggregate.translateToPlan(DataStreamGroupAggregate.scala:113)
    at org.apache.flink.table.plan.nodes.datastream.DataStreamCalc.translateToPlan(DataStreamCalc.scala:97)
    at org.apache.flink.table.api.StreamTableEnvironment.translateToCRow(StreamTableEnvironment.scala:837)
    at org.apache.flink.table.api.StreamTableEnvironment.translate(StreamTableEnvironment.scala:764)
    at org.apache.flink.table.api.StreamTableEnvironment.translate(StreamTableEnvironment.scala:734)
    at org.apache.flink.table.api.java.StreamTableEnvironment.toRetractStream(StreamTableEnvironment.scala:414)
    at org.apache.flink.table.api.java.StreamTableEnvironment.toRetractStream(StreamTableEnvironment.scala:357)

如何实现此查询？




Answer 1

匿名用户

更新：Flink 1.6.0可用，支持流表上的DISTINCT聚合。

flink(1.4 . x版)尚不支持流表上具有< code>DISTINCT聚合的SQL查询。支持的目标是2018年年中之前不会发布的Flink 1.6。

但是，您可以实现一个用户定义的聚合函数来计算不同的计数，并在注册后在查询中使用该函数。查询语法当然会有所不同。