当前位置: 首页 > 知识库问答 >
问题:

Avro映射类型字段不支持默认值

曹臻
2023-03-14

avro.version=

在avro模式中,map type不支持缺省值。我尝试了以下不同的模式。

第一:

{
        "name": "constrainedQuantities",
        "doc": "Map<Long, Integer>; In converter, we will map key of type String to Long type",
        "type": {
          "type": "map",
          "values": ["null","int"],
          "default": null
        }
      }

第二:

{
        "name": "constrainedQuantities",
        "doc": "Map<Long, Integer>; In converter, we will map key of type String to Long type",
        "type": {
          "type": "map",
          "values": ["null","int"]
        }
      }

第三:

{
        "name": "constrainedQuantities",
        "doc": "Map<Long, Integer>; In converter, we will map key of type String to Long type",
        "type": {
          "type": "map",
          "values": "int",
          "default": null
        }
      }

Java代码如下:

我只希望constraintQuantities和RawQuanties为null。因为这些是可选字段。即使我没有将它们设置为null,它也会抛出异常。

AvroEdfWorkflowStatus workflowStatus = AvroEdfWorkflowStatus.newBuilder()
                .setRunUUID("2c9967ff-fd35-4c37-a597-8a6fa5360bf6")
                .setEffDate(LocalDate.of(2008, 06, 9).toString())
                .setGroupId(704801)
                .setSourceLoc(7048)
                .setPriority("P0")
                .setDestinationLoc(1554)
                .setStatus("COMPLETED")
                .setItem("9203427")
                .setConstrainedQuantities(null)
                .setRawQuantities(null)
                .build();

最重要的是,方法生成java POJO,但无法构建该对象。

org.apache.avro.AvroRuntimeException: Field constrainedQuantities type:MAP pos:8 does not accept null values
    at org.apache.avro.data.RecordBuilderBase.validate(RecordBuilderBase.java:95)
    at com.example.AvroEdfWorkflowStatus$Builder.setConstrainedQuantities(AvroEdfWorkflowStatus.java:818)
    at com.codenotfound.kafka.ApiController.sendWorkflowStatusTopic(ApiController.java:113)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:108)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1457)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

如果不设置它们为空,然后接收以下异常:

    at org.apache.avro.generic.GenericData.getDefaultValue(GenericData.java:1151)
    at org.apache.avro.data.RecordBuilderBase.defaultValue(RecordBuilderBase.java:142)
    at com.example.AvroEdfWorkflowStatus$Builder.build(AvroEdfWorkflowStatus.java:902)
    at com.codenotfound.kafka.ApiController.sendWorkflowStatusTopic(ApiController.java:113)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:108)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1457)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

有人能帮忙吗?

共有1个答案

轩辕乐邦
2023-03-14

看起来您希望ConstrainedQuantilities成为记录中的一个可选字段(即null是该字段的有效值)。

如果是这样的话,这可能就是你要找的:

{
  "name": "AvroEdfWorkflowStatus",
  "fields": [
    {
      "name": "constrainedQuantities",
      "type": [ "null",
        {
          "type": "map",
          "values": "int"
        }
      ],
      "default": null
    }
  ]
}

一些注意事项:

  • 在这种情况下,字段现在可以包含一个空值,您的构建者应该很高兴不设置它,或将其设置为空,或将其设置为Map

 类似资料:
  • 问题内容: 我在设置Avro字段的默认值时遇到了一些问题。我有一个简单的架构,如下所示: data.avsc: 我正在使用 avro-maven-plugin v1.7.6 生成Java模型。 当我使用:创建模型的实例时 ,它会失败并出现以下异常: org.apache.avro.AvroRuntimeException:org.apache.avro.AvroRuntimeException:字

  • 我在设置Avro字段的默认值时遇到了一些问题。我有一个简单的模式,如下所示: 数据avsc: 我使用的是avro maven插件v1。7.6生成Java模型。 当我使用:创建模型的实例时,它会失败,但有一个异常: org.apache.avro.AvroRuntimeExc0019:org.apache.avro.AvroRuntimeExc0019:字段id类型: UNION pos: 0未设置

  • 我正在尝试使用Flink的SQL API从地图访问密钥。它失败,线程“main”组织中出现错误异常。阿帕奇。Flink。桌子应用程序编程接口。TableException:不支持类型:任何请告知我如何修复它。这是我的活动课 这是提交flink作业的主类 当我运行它时,我得到了例外 我正在使用flink 1.3.1

  • 我正在尝试序列化一个avro模式,然后将其写入hortonworks模式注册表,但在写入操作期间收到以下错误消息。 这是avro模式 无论谁,如果我添加“null”作为联合的第一种类型,都会成功。avro联合类型需要“null”吗?在我的情况下,这将是不正确的数据表示,所以我不热衷于这样做。如果有什么不同,我使用avro 1.9.1。 此外,如果标签不正确,但找不到标签并且没有足够的代表来创建新标

  • 我的服务器资源: 我的客户资源: 我的Build.Gradle文件:

  • 在我的搜索引擎中,用户可以选择搜索大小写敏感或不敏感。如果他们选择这样做,查询将搜索使用自定义区分大小写分析器的字段。这是我的设置: 获取/candidates/_settings 因此,我从这个答案中创建了一个名为的自定义分析器。我试图定义我的,如下所示: 把/candidates/_mapping/candidate 因此,在查询区分大小写的匹配时,我可以执行以下操作: 我甚至没有进入最后一步