Spring批处理：如何配置FlatItemFileReader来读取压缩的ND-JSON文件？

Question

{"summersolstice":"21st June, 2019", //complex objects, arrays, etc ... }
{"summersolstice":"21st June, 2018", //complex objects, arrays, etc ... }
{"summersolstice":"21st June, 2017", //complex objects, arrays, etc ... }

虽然我知道最好的方法是使用去年夏天发布的闪亮的JSON item reader devs(link)，但要将批处理版本更新到最新版本还不可行。这是唯一的限制。

到目前为止，我遵循了这个堆栈答案，但我不认为将FlatFileItemReader作为Map 使用T是最佳策略！到目前为止，我只是用下面的代码来理解它：

public class JsonItemReader extends FlatFileItemReader<Map<String, Object>> {

    public JsonItemReader(File file) {
        Resource resource = new BzipLazyResource(file); //on the fly unzipping
        setResource(resource);
        setLineMapper(new JsonLineMapper());
    }

    public JsonItemReader(String sourceFileName) {
        this(new File(sourceFileName));
    }
}

...然后在项目处理器中简单地解析它，比如：

public class JsonItemProcessor implements ItemProcessor<Map<String, Object>, List<Json>> {

    private ObjectMapper mapper = new ObjectMapper();
    private static final Logger logger = LoggerFactory.getLogger(JsonItemProcessor.class);

    public List<Json> process(Map<String, Object> jsonItem) throws Exception {
        JsonNode jsonNode = mapper.valueToTree(jsonItem);
        return parseJsonItems(jsonNode);
    }

• Spring Batch：如何设置FlatItemFileReader以读取JSON文件？
• 如何使用Jackson将HashMap转换为JsonNode
• 新行分隔JSON
• 如何实时读取压缩文件
• 批处理4.1.x JsonItemReader

Answer 1

通过扩展FlatFileItemReader并创建一个附加的项处理器来将映射 转换为列表 ，就不需要这种间接性了。

下面是一个可以与FlatFileItemReader一起使用的快速行映射器：

import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.batch.item.file.LineMapper;

public class NDJsonLineMapper<T> implements LineMapper<T> {

    private Class<? extends T> targetType;

    private ObjectMapper objectMapper = new ObjectMapper(); // TODO could make this configurable

    public NDJsonLineMapper(Class<? extends T> targetType) {
        this.targetType = targetType;
    }

    @Override
    public T mapLine(String line, int lineNumber) throws Exception {
        return objectMapper.readValue(line, targetType);
    }
}

及其测试：

import org.junit.Assert;
import org.junit.jupiter.api.Test;

class NDJsonLineMapperTest {

    @Test
    void testNDJsonMapping() throws Exception {
        // given
        String jsonLine = "{\"id\":1,\"name\":\"foo\"}";
        NDJsonLineMapper<Person> lineMapper = new NDJsonLineMapper<>(Person.class);

        // when
        Person person = lineMapper.mapLine(jsonLine, 1);

        // then
        Assert.assertEquals(1, person.getId());
        Assert.assertEquals("foo", person.getName());

    }

    static class Person {

        private int id;
        private String name;

        public int getId() {
            return id;
        }

        public void setId(int id) {
            this.id = id;
        }

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }
    }

}

@Bean
public FlatFileItemReader<Person> itemReader() {
    return new FlatFileItemReaderBuilder<Person>()
            .name("NDJsonItemReader")
            .resource(new FileSystemResource("file.ndjson"))
            .lineMapper(new NDJsonLineMapper<>(Person.class))
            .build();
}