具有Jackson/解析性能的Json数组扁平化

Question

我有一个如下所示的JSON：我的目标POJO是

[{
        "id": "1",
        "teams": [{
                "name": "barca"
            },
            {
                "name": "real"
            }
        ]
    },
    {
        "id": "2"
    },
    {
        "id": "3",
        "teams": [{
                "name": "atletico"
            },
            {
                "name": "cz"
            }
        ]
    }
]

我的目标是

class Team
int id;
String name;

new Team(1,barca)
new Team(1,real)
new Team(2,null)
new Team(3,atletico)
...

            JsonNode rootArray = jsonParser.readValueAsTree();
            for (JsonNode root : rootArray) {
                String id = root.get("id").toString();
                JsonNode teamsNodeArray = root.get("teams");
                if (teamsNodeArray != null) {
                    for (JsonNode teamNode: teamsNodeArray ) {
                        String nameString = teamNode.get("name").toString();
                        teamList.add(new Team(id, nameString));
                    }
                } else {
                    teamList.add(new Team(id, null));
                }
            }

提前谢谢你。

Answer 1

不要自己解析数据，尽可能使用自动解/序列化。

使用jackson可以非常简单：

MyData myData = new ObjectMapper().readValue(rawData, MyData.class);

对于您的特定示例，我们生成了一个非常大的实例(10M行）：

$ head big.json 
[{"id": 1593, "group": "6141", "teams": [{"id": 10502, "name": "10680"}, {"id": 16435, "name": "18351"}]}
,{"id": 28478, "group": "3142", "teams": [{"id": 30951, "name": "3839"}, {"id": 25310, "name": "19839"}]}
,{"id": 29810, "group": "8889", "teams": [{"id": 5586, "name": "8825"}, {"id": 27202, "name": "7335"}]}
...
$ wc -l big.json 
10000000 big.json

public static class Team {
    public int id;
    public String name;
}

public static class Group {
    public int id;
    public String group;
    public List<Team> teams;
}

List<Group> xs = new ObjectMapper()
                   .readValue(
                       new File(".../big.json"),
                       new TypeReference<List<Group>>() {});

public static void main(String... args) throws IOException {

    long t0 = System.currentTimeMillis();

    List<Group> xs = new ObjectMapper().readValue(new File("/home/josejuan/tmp/1/big.json"), new TypeReference<List<Group>>() {});

    long t1 = System.currentTimeMillis();

    // test: add all group id
    long groupIds = xs.stream().mapToLong(x -> x.id).sum();

    long t2 = System.currentTimeMillis();

    System.out.printf("Group id sum := %d, Read time := %d mS, Sum time = %d mS%n", groupIds, t1 - t0, t2 - t1);
}

Group id sum := 163827035542, Read time := 10710 mS, Sum time = 74 mS

$ perl -n -e 'print "$1\n" if /"id": ([0-9]+), "group/' big.json | time awk '{s+=$1}END{print s}'
163827035542
4.96user

处理数据的非性能问题可以通过多种方式解决，具体取决于您希望如何使用这些信息。例如，可以将所有设备分组：

List<Team> teams = xs.stream()
                     .flatMap(x -> x.teams.stream())
                     .collect(toList());

Map<Integer, Team> uniqTeams = xs.stream()
                                 .flatMap(x -> x.teams.stream())
                                 .collect(toMap(
                                      x -> x.id,
                                      x -> x,
                                      (a, b) -> a));