使用泛型类型为Java POJO生成Avro模式

Question

我正在尝试使用以下方法在运行时获取Avro模式：

private Schema getSchema(Class clazz) {
    Schema s = ReflectData.get().getSchema(clazz);
    AvroSchema avroSchema = new AvroSchema(s);
    return avroSchema.getAvroSchema();
  }

但由于我的POJO类包含以下泛型：

public abstract class Data<T> implements Serializable {
    private static final long serialVersionUID = 1L;
    private String dataType;
    private T id;

    public Data() {
    }

    public Data(String dataType) {
        this.dataType = dataType;
    }

    public Data(String dataType, T id) {
        this.dataType = dataType;
        this.id = id;
    }
}

我得到以下例外情况：

Exception in thread "main" org.apache.avro.AvroRuntimeException: avro.shaded.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.avro.AvroTypeException: Unknown type: T
    at org.apache.avro.specific.SpecificData.getSchema(SpecificData.java:227)

我知道Avro不会支持泛型类型。是否有一种方法可以在运行时生成架构时从类中省略某些类字段？

Answer 1

private <T> String writePojoToParquet(List<T> pojos, String fileKey){
        String fileName = fileKey + ".parquet";
        Path path = new Path(fileName.replace("/", "_"));
        //No matter what delete file always.
        String strPath = path.toString();
        FileUtils.delete(strPath);
        FileUtils.delete(strPath + ".crc");
        logger.debug("Writing data to parquet file {}", strPath);
        Configuration conf = new Configuration();
        try (ParquetWriter<T> writer =
                     AvroParquetWriter.<T>builder(path)
                             .withSchema(ReflectData.AllowNull.get().getSchema(pojos.get(0).getClass()))
                             .withDataModel(ReflectData.get())
                             .withConf(conf)
                             .withCompressionCodec(CompressionCodecName.SNAPPY)
                             .withWriteMode(ParquetFileWriter.Mode.OVERWRITE)
                             .enableValidation()
                             .enableDictionaryEncoding()
                             .build()) {
            for (T p : pojos) {
                writer.write(p);
            }
            return strPath;
        } catch (IOException e) {
            logger.error("Error while writing data to parquet file {}.", strPath, e);
        }
        return null;
    }