如何从GenericRecord获取类型化值？

Question

我正在与Avro合作，我有一个通用记录。我想从中提取clientId、deviceName、holder。在Avro模式中，clientId是整数，deviceName是字符串，holder是映射。

avro模式中的clientId：

{
    "name" : "clientId",
    "type" : [ "null", "int" ],
    "doc" : "hello"
}

avro架构中的设备名称：

{
    "name" : "deviceName",
    "type" : [ "null", "string" ],
    "doc" : "test"
}

avro模式中的持有人：

{
    "name" : "holder",
    "type" : {
      "type" : "map",
      "values" : "string"
    }
}

我的问题是-检索类型化值的推荐方法是什么，而不是对象？

在下面的代码中，有效负载是GenericRecord，我们可以从中获得avro模式。这就是我现在正在做的，将所有内容提取为字符串。但我如何才能得到类型化的值呢。有什么办法吗？我的意思是，无论avro模式中的数据类型是什么，我只想提取它。

  public static void getData(GenericRecord payload) {
    String id = String.valueOf(payload.get("clientId"));
    String name = String.valueOf(payload.get("deviceName"));

    // not sure how to get maps here
  }

所以我想提取clientId作为整数，deviceName作为字符串和持有者作为JavamapMap




Answer 1

您可以尝试这种方法。对于健壮的实现，您应该考虑使用模式编译生成代码。

package stackoverflow;

import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertThat;

import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;

import org.apache.avro.AvroTypeException;
import org.apache.avro.Schema;
import org.apache.avro.Schema.Field;
import org.apache.avro.Schema.Type;
import org.apache.avro.generic.GenericData.Record;
import org.apache.avro.generic.GenericRecord;
import org.apache.avro.util.Utf8;
import org.junit.Test;

// Just for demonistration; not robust implementation
public class GenericRecordType {
    @Test
    public void testName() throws Exception {
        Schema schema = buildSchema();

        GenericRecord record = new Record(schema);
        record.put("clientId", 12);
        record.put("deviceName", "GlassScanner");
        record.put("holder", new HashMap<>());

        Integer value = IntField.clientId.getValue(record);
        String deviceName = StringField.deviceName.getValue(record);
        Map<String, String> mapString = MapOfStringField.holder.getValue(record);

        assertThat(deviceName, is("GlassScanner"));
        assertThat(value, is(12));
        assertThat(mapString.size(), is(0));
    }

    private Schema buildSchema() {
        Field clientId = new Field("clientId", Schema.create(Type.INT), "hello", (Object) null);
        Field deviceName = new Field("deviceName", Schema.create(Type.STRING), "hello", (Object) null);
        Field holder = new Field("holder", Schema.createMap(Schema.create(Type.STRING)), null, (Object) null);
        Schema schema = Schema.createRecord(Arrays.asList(clientId, deviceName, holder));
        return schema;
    }

    public static interface TypedField<T> {
        String name();

        public T getValue(GenericRecord record);

    }

    public static enum StringField implements TypedField<String> {
        deviceName;

        @Override
        public String getValue(GenericRecord record) {
            String typed = null;
            Object raw = record.get(name());
            if (raw != null) {
                if (!(raw instanceof String || raw instanceof Utf8)) {
                    throw new AvroTypeException("string type was epected for field:" + name());
                }
                typed = raw.toString();
            }
            return typed;
        }

    }

    public static enum IntField implements TypedField<Integer> {
        clientId;

        private IntField() {
        }

        @Override
        public Integer getValue(GenericRecord record) {
            Integer typed = null;
            Object raw = record.get(name());
            if (raw != null) {
                if (!(raw instanceof Integer)) {
                    throw new AvroTypeException("int type was epected for field:" + name());
                }
                typed = (Integer) raw;
            }
            return typed;
        }

    }

    public static enum MapOfStringField implements TypedField<Map<String, String>> {
        holder;

        @Override
        @SuppressWarnings("unchecked")
        public Map<String, String> getValue(GenericRecord record) {
            Map<String, String> typed = null;
            Object raw = record.get(name());
            if (raw != null) {
                if (!(raw instanceof Map)) {
                    throw new AvroTypeException("map type was epected for field:" + name());
                }
                typed = (Map<String, String>) raw;
            }
            return typed;
        }
    }

}

Answer 2

您应该能够将您的string值转换为Utf8，int转换为整数，以及map到Map

public static void getData(GenericRecord payload) {
    int id = (Integer) payload.get("clientId");
    String name = payload.get("deviceName").toString(); // calls Utf8.toString
    Map<Utf8, Utf8> holder = (Map<Utf8, Utf8>) payload.get("holder");

    ...
}

总的来说，我相信你可以做这些演员：

• 原语成为其装箱版本（整数、双精度等）
• 字符串变为Utf8
• 字节变为java。nio。ByteBuffer
• 数组变为java。util。集合
• 映射成为java。util。地图