elasticsearch不响应批量操作

Question

我在Azure上有3个主节点+3个数据节点elasticsearch集群。我试图执行批量操作，但我得到了关于节点本身的失败错误，以下是我如何设置客户端：

    final Builder builder = Settings.builder();
    final org.elasticsearch.client.transport.TransportClient.Builder transBuilder = TransportClient.builder();
    builder.put("cluster.name", esCluster);
    if (esShield) {
        builder.put("shield.user", esUsername + ":" + esPassword);
        transBuilder.addPlugin(ShieldPlugin.class);
    }
    final Settings settings = builder.build();
    TransportClient esClient = transBuilder.settings(settings).build();
    final String[] hosts = esHost.split(",");
    for (String host : hosts) {
        esClient.addTransportAddress(new InetSocketTransportAddress(new InetSocketAddress(host, Integer.parseInt(esPort))));
    }

下面是批量操作：

BulkProcessor bulkProcessor = BulkProcessor.builder(getClient(), new BulkProcessor.Listener() {
        @Override
        public void beforeBulk(long executionId, BulkRequest request) {
            LOGGER.info("Going to execute new bulk composed of {" + request.numberOfActions() + "} actions");
        }

        @Override
        public void afterBulk(long executionId, BulkRequest request, BulkResponse response) {
            LOGGER.info("Executed bulk composed of {" + request.numberOfActions() + "} actions");
        }

        @Override
        public void afterBulk(long executionId, BulkRequest request, Throwable failure) {
            LOGGER.info("Error executing bulk");
            failure.printStackTrace();
        }
    }).setBulkActions(docs.size()).setConcurrentRequests(250).build();
    for (DBObject doc : docs) {
        bulkProcessor.add(getClient().prepareIndex(indexName, typeName).setSource(doc.toMap()).request());
    }

它开始对1000个记录批次做出罚款响应，如下所示：

将执行由{1001}个操作组成的新大容量

然后我开始得到以下错误：

传输：383-[Stanley Stewart]未能获取{#传输#-1}{10.0.0.10}{10.0.0.10:9300}的节点信息，正在断开连接...ReceiveTimeOutTransportException[[][10.0.0.10:9300][Cluster:Monitor/Nodes/Livential]request_id[60]在[5000ms]之后超时]在org.ellasticSearch.transport.transportService$timeouthandler.run(TransportService.java:679)在java.util.concurrent.threadPoolExecutor.runworker(threadPoolExecutor.java:1142)在

最后我得到了以下错误：

大容量：148-[Stanley Stewart]无法执行大容量请求%1。nonodeAvailableException[nonodeAvailableException[nonodeAvailable[nonodeAvailable][nonodeAvailable[nonodeAvailable][nonodeAvailable][nonodeAvailable][nonodeAvailable][nonodeAvailable][没有配置的节点可用:[{#Transport#-1}{10.0.0.10}{10.0.0.10:9300}，{#Transport#-2}{10.0.0.11}{10.0.0.11:9300}，{#Transport#-3}{port.transportProxyClient.execute(TransportProxyClient.java:55)在org.ellasticSearch.client.transport.transportClient.doExecute(TransportClient.java:288)在org.ellasticSearch.client.support.abstractClient.execute(abstractClient.java:359)在org.ellasticSearch.client.support.abstractClient.bulk(abstractClient.java:436)在.action.bulk.bulkreque在org.elasticsearch.action.bulk.bulkprocessor.execute(bulkprocessor.java:121)在org.elasticsearch.action.bulk.bulkprocessor.execute(bulkprocessor.java:312)在org.elasticsearch.action.bulk.bulkprocessor.executeIfneed(bulkprocessor.java:303)在org.elasticsearch.action.bulk.processor.internaladd(bulkprocessor.java:285)在4）在org.elasticsearch.action.bulk.bulkprocessor.add(bulkprocessor.java:250)

谁能帮我弄清楚是怎么回事，怎么解决？

Answer 1

这可能是因为索引的刷新间隔太低。尝试在大容量处理之前将索引的刷新间隔设置为-1。批量处理完成后可以重置它。

https://www.elastic.co/guide/en/elasticsearch/reference/current/indexes-update-settings.html#bulk