将 Elasticsearch 添加到 “Spring Data MongoDB Project”

Question

第二个问题：当elastic和mongo都有@Documnet注释时，如何注释模型类？现在我得到一个错误，认为它与这个问题有关：“找不到类的id属性”

@Document(indexName = "gold", type = "article")
@Document
public class GoldSnapShot {
    public static final String TOPIC = "goldsnapshot";
    Double ons;
    Double gram18;
    Double irec;
    Double dollar;
    @org.springframework.format.annotation.DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
    private DateTime timestamp;

最后，我将感谢任何关于最佳实践和更干净的编码的指导。

Spring Boot应用程序：

@EnableMongoRepositories("com.sames.samesgoldconsumer.Repository")
@EnableElasticsearchRepositories(basePackages = "com.sames.samesgoldconsumer.elasticsearch.Repository")
@SpringBootApplication
public class SamesGoldConsumerApplication {

    public static void main(String[] args) {
        SpringApplication.run(SamesGoldConsumerApplication.class, args);
    }

}

Kafka消费者服务：

@Component
public class KafkaConsumerService {
    private final GoldSnapShotMongoRepository goldSnapShotMongoRepository;
    private final GoldSnapShotElasticRepository goldSnapShotElasticRepository;

    private final Logger logger
            = LoggerFactory.getLogger(KafkaConsumerService.class);

    public KafkaConsumerService(GoldSnapShotMongoRepository goldSnapShotMongoRepository,
                                GoldSnapShotElasticRepository goldSnapShotElasticRepository) {
        this.goldSnapShotMongoRepository = goldSnapShotMongoRepository;
        this.goldSnapShotElasticRepository = goldSnapShotElasticRepository;
    }
    @KafkaListener(topics = "gold", groupId = AppConstants.GROUP_ID)
    public void processMessage(GoldSnapShot goldSnapShot) {
        logger.info(String.format("GoldSnapShot Received -> %s", goldSnapShot.toString()));
        goldSnapShotMongoRepository.save(goldSnapShot);
        goldSnapShotElasticRepository.save(goldSnapShot);
    }
}

弹性配置：

@Configuration
public class ElasticConfig {

    @Bean
    public RestHighLevelClient client() {
        ClientConfiguration clientConfiguration
                = ClientConfiguration.builder()
                .connectedTo("localhost:9200")
                .build();

        return RestClients.create(clientConfiguration).rest();
    }

    @Bean
    public ElasticsearchOperations elasticsearchTemplate() {
        return new ElasticsearchRestTemplate(client());
    }
}

Kafka 配置：

@EnableKafka
@Configuration
public class KafkaConfig {
//    @Value("${spring.kafka.consumer.group-id}")
//    private String brokers;
    private final String brokers = "localhost:9092";
//    @Value("${spring.kafka.bootstrap-servers}")
//    private String groupId;
    private final String groupId = "group-id";
    @Bean
    public ConsumerFactory<String, GoldSnapShot> consumerFactory() {
        Map<String, Object> props = new HashMap<>();
        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, brokers);
        props.put(ConsumerConfig.GROUP_ID_CONFIG, groupId);
        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ErrorHandlingDeserializer.class);
        props.put(ErrorHandlingDeserializer.VALUE_DESERIALIZER_CLASS, JsonDeserializer.class);
        props.put(JsonDeserializer.VALUE_DEFAULT_TYPE, GoldSnapShot.class);
        return new DefaultKafkaConsumerFactory<>(props);
    }
    @Bean
    KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<String, GoldSnapShot>>
    kafkaListenerContainerFactory() {
        ConcurrentKafkaListenerContainerFactory<String, GoldSnapShot> factory =
                new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(consumerFactory());
        return factory;
    }
}

Answer 1

你真的需要在ES中存储整个对象吗？我只是想知道你能在es中做什么MongoDB不提供的搜索？数据中没有文本字段，只有数值和日期。

我强烈建议不要将同一类用于3个目的：从Kafka读取、存储到MongoDB和Elasticsearch。为SpringData模块定义单独的类，并编写映射器，将从Kafka读取的数据转换为单独的数据模型。

否则，您将在类上有两个不同的@Document注释，其中一个需要完全限定。

至于 Elasticsearch 部分：您显然使用的是过时的 Spring Data Elasticsearch，在 4.1 版中删除了 @Document 的类型参数。注解 @org.springframework.format.annotation.DateTimeFormat 在 SDE 中不起作用，您需要向该属性添加具有适当的 Elasticsearch 支持的日期时间格式的@Field注解。

如错误所示，您需要提供一个用< code>@Id注释或命名为< code>id的属性，该属性包含该对象的唯一标识符(它可能为空，然后Elasticsearch将为此创建一个唯一的字符串值)。