Spring批处理JpaPagingItemReader页大小和maxitemcount

Question

我使用JpaPagingItemReader和Spring批处理作业从数据库中读取数据。我有一个要求，我只能写10K记录到外部服务器在每次尝试。我将块大小和页面大小设置为10000。这是否有助于在每次尝试中只读取、处理和写入10K记录，直到JpaPagingItemReader以响应null耗尽并读取来自DB的所有数据为止？我混淆了Pagesize和setMaxItemCount值。哪种方法可以确保从DB读取10K记录？

批配置步骤：

    public Step step1(StepBuilderFactory stepBuilderFactory, @Qualifier("itemReader") ItemReader<BatchRecords> ir,
            @Qualifier("itemProcessor") ItemProcessor<BatchRecords, Map<String, Object>> ip,
            @Qualifier("itemWriter") ItemWriter<Map<String, Object>> iw) throws Exception {
        logger.info("Step1");
        return stepBuilderFactory.get("step1").<BatchRecords, Map<String, Object>>chunk(10000).reader(ir).processor(ip)
                .writer(iw).build();

    }



ItemReader:

    @Bean(destroyMethod="")
    @StepScope
    public JpaPagingItemReader<BatchRecords> itemReader(){
        Map<String, Object> dates = batchUtil.dateFormat();
        String startDate = (String) dates.get("StartDate");
        String endDate = (String) dates.get("EndDate");
        logger.info("Batch settlement Start date : {}, End Date: {}",startDate,endDate);
        JpaPagingItemReader<BatchRecords> reader = new JpaPagingItemReader<BatchRecords>();
        reader.setEntityManagerFactory(localContainerdbEntityManagerFactory.getNativeEntityManagerFactory());
        reader.setQueryString("select b from BatchRecords b where b.status = 'O' and b.ordTimestamp between '" + startDate + "' and '" + endDate+ "' ");

        reader.setPageSize(10000);
//      reader.setMaxItemCount(50*reader.getPageSize());
        try {
            reader.afterPropertiesSet();
        } catch (Exception e) {
            logger.error("Exception in paging Item reader: "+e.getMessage());
        }
        return reader;

    }```

Answer 1

如果要限制要读取的项总数，则需要使用maxItemCount。我认为在您的SQL查询中设置限制更好。pageSize将告诉读者每页要阅读多少项（这不是对项总数的限制，而是对请求新页时要阅读的项数的限制）。