如何在Elasticsearch的聚合中添加分页

Question

我有一个elasticsearch请求如下：

{
    "size":0,
    "aggs":{
        "group_by_state":{
            "terms":{
                "field":"poi_id"
            },
            "aggs":{
                "sum(price)":{
                    "sum":{
                        "field":"price"
                    }
                }
            }
        }
    }
}

我想在这个请求中添加分页，就像

select poi_id, sum(price) from table group by poi_id limit 0,2

我搜索了很多，找到了一个关于它的链接：https://github.com/elastic/elasticsearch/issues/4915.

但我仍然没有得到实现方法。

有没有办法通过Elasticsearch本身而不是我的应用程序来实现它？

Answer 1

您可以在请求中使用来自和大小的参数。有关更多信息，请参阅https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-from-size.html。您的请求如下所示：

{
    "from" : 0, 
    "size" : 10,
    "aggs":{
        "group_by_state":{
            "terms":{
                "field":"poi_id"
            },
            "aggs":{
                "sum(price)":{
                    "sum":{
                        "field":"price"
                    }
                }
            }
        }
    }
}

Answer 2

晚到派对，但刚刚在v6.3中发现了“复合”聚合。这些允许：
1。更像“Sql”的分组
2。使用“after\u键”分页<拯救了我们的一天，希望它也能帮助别人。

例如，在2个日期之间获得每小时命中视频数，按5个字段分组：

GET myindex-idx/_search
{
  "query": {
    "bool": {
      "must": [
        {"match": {"docType": "myDOcType"}}, 
        {"range": {
          "@date": {"gte": "2019-06-19T21:00:00", "lt": "2019-06-19T22:00:00"}
          }
        }
      ]
    }
  }, 
  "size": 0, 
  "aggs": {
    "mybuckets": {
      "composite": {
        "size": 100, 
        "sources": [
          {"@date": {
            "date_histogram": {
              "field": "@date", 
              "interval": "hour", 
              "format": "date_hour"}
            }
          }, 
          {"field_1": {"terms": {"field": "field_1"}}}, 
          {"field_2": {"terms": {"field": "field_2"}}}, 
          {"field_3": {"terms": {"field": "field_3"}}}, 
          {"field_4": {"terms": {"field": "field_4"}}}, 
          {"field_5": {"terms": {"field": "field_5"}}}
        ]
      }
    }
  }
}

产生：

{
  "took": 255,
  "timed_out": false,
  "_shards": {
    "total": 80,
    "successful": 80,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 46989,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "mybuckets": {
      "after_key": {
        "@date": "2019-06-19T21",
        "field_1": 262,
        "field_2": 347,
        "field_3": 945,
        "field_4": 2258,
        "field_5": 0
      },
      "buckets": [
        {
          "key": {
            "@date": "2019-06-19T21",
            "field_1": 56,
            "field_2": 106,
            "field_3": 13224,
            "field_4": 46239,
            "field_5": 0
          },
          "doc_count": 3
        },
        {
          "key": {
            "@date": "2019-06-19T21",
            "field_1": 56,
            "field_2": 106,
            "field_3": 32338,
            "field_4": 76919,
            "field_5": 0
          },
          "doc_count": 2
        },
        ....

下面是这样发出的分页查询，使用“查询中的after\u key对象”after对象：

GET myindex-idx/_search
{
  "query": {
    "bool": {
      "must": [
        {"match": {"docType": "myDOcType"}}, 
        {"range": {
          "@date": {"gte": "2019-06-19T21:00:00", "lt": "2019-06-19T22:00:00"}
          }
        }
      ]
    }
  }, 
  "size": 0, 
  "aggs": {
    "mybuckets": {
      "composite": {
        "size": 100, 
        "sources": [
          {"@date": {
            "date_histogram": {
              "field": "@date", 
              "interval": "hour", 
              "format": "date_hour"}
            }
          }, 
          {"field_1": {"terms": {"field": "field_1"}}}, 
          {"field_2": {"terms": {"field": "field_2"}}}, 
          {"field_3": {"terms": {"field": "field_3"}}}, 
          {"field_4": {"terms": {"field": "field_4"}}}, 
          {"field_5": {"terms": {"field": "field_5"}}}
        ],
      "after": {
        "@date": "2019-06-19T21",
        "field_1": 262,
        "field_2": 347,
        "field_3": 945,
        "field_4": 2258,
        "field_5": 0
        }
      }
    }
  }
}

这将遍历结果，直到mybuckets返回为空

Answer 3

目前，我正在研究分页聚合结果的解决方案。您要使用的是分区。官方文件中的这一部分非常有用。https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_filtering_values_with_partitions

为了调整您的示例，术语设置将更新如下。

{
    "size":0,
    "aggs":{
        "group_by_state":{
            "terms":{
                "field":"poi_id",
                "include": {
                    "partition": 0,
                    "num_of_partitions": 100
                },
                "size": 10000
            },
            "aggs":{
                "sum(price)":{
                    "sum":{
                        "field":"price"
                    }
                }
            }
        }
    }
}

这将把你的结果分成100个分区（num_of_partitions），每个分区的最大大小为10k个结果（size），并检索第一个这样的分区（分区： 0）

如果要聚合的字段有超过10k个唯一值（并希望返回所有值），则需要增加值，或根据字段的基数动态计算分区的大小和数量。https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-cardinality-aggregation.html#search-聚合度量基数聚合

您可能还想使用show\u term\u doc\u count\u error设置来确保聚合返回准确的计数。https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_per_bucket_document_count_error

希望这有帮助。