我使用的是带有Kafka活页夹和Avro的SpringCloudStream版本2.2.0。显然,一个不正确的记录被发布到Kafka主题中,导致所有消费者返回反序列化错误,并进行某种无限重试。
Error deserializing key/value for partition realtime-object-1 at offset 908. If needed, please seek past the record to continue consumption. Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 13 Caused by: org.apache.kafka.common.errors.SerializationException: Could not find class au.com.brolly.avro.model.GenericTaskTriggerCapsule specified in writer's schema whilst finding reader's schema for a SpecificRecord.
从技术上讲,应该有一种方法可以指定反序列化异常的策略。我可以找到一些不同的策略,如logAnd继续
和sendToDlq
,但它们适用于我在应用程序中不使用的Kafka流。如果有人能帮助我理解这里缺少什么,我将不胜感激。
请参阅spring-kafka文档。
使用Error Handling Des身旁化器2
来包装avro反序列化器。异常将被发送到错误处理程序(默认情况下,它只会记录错误)。
您可以使用ListenerContainerCustomizer
bean添加SeekToCurrentErrorHandler
(可选使用DeadLetterPublishingRecover
),将反序列化失败的记录发送到另一个主题。
编辑
这是一个使用Spring Cloud Stream配置它的示例…
@SpringBootApplication
@EnableBinding(Sink.class)
public class So60827393Application {
public static void main(String[] args) {
SpringApplication.run(So60827393Application.class, args);
}
@StreamListener(Sink.INPUT)
public void listen(String in) {
}
@Bean
public ApplicationRunner runner(KafkaTemplate<byte[], byte[]> template) {
return args -> template.send("foo", "foo".getBytes());
}
}
public class MyBadDeserializer implements Deserializer {
@Override
public Object deserialize(String topic, byte[] data) {
throw new RuntimeException("Always fails");
}
}
spring.cloud.stream.bindings.input.destination=foo
spring.cloud.stream.bindings.input.group=foo
spring.cloud.stream.bindings.input.consumer.use-native-decoding=true
spring.cloud.stream.kafka.bindings.input.consumer.configuration.value.deserializer=org.springframework.kafka.support.serializer.ErrorHandlingDeserializer2
spring.cloud.stream.kafka.bindings.input.consumer.configuration.spring.deserializer.value.delegate.class=com.example.demo.MyBadDeserializer
如前所述,向配置中添加任何其他属性(如模式注册表URL)(EHD2配置其委托反序列化程序)。
2020-03-24 18:18:35.633 ERROR 61822 --- [container-0-C-1] o.s.kafka.listener.LoggingErrorHandler : Error while processing: ConsumerRecord(topic = foo, partition = 0, leaderEpoch = 0, offset = 0, CreateTime = 1585088312600, serialized key size = -1, serialized value size = 3, headers = RecordHeaders(headers = [RecordHeader(key = springDeserializerExceptionValue, value = [-84, -19, 0, 5, 115, 114, 0, 69, 111, 114, 103, 46, 115, 112, 114, 105, 110, 103, 102, 114, 97, 109, 101, 119, 111, 114, 107, 46, 107, 97, 102, 107, 97, 46, 115, 117, 112, 112, 111, 114, 116, 46, 115, 101, 114, 105, 97, 108, 105, 122, 101, 114, 46, 68, 101, 115, 101, 114, 105, 97, 108, 105, 122, 97, 116, 105, 111, 110, 69, 120, 99, 101, 112, 116, 105, 111, 110, -26, -50, 105, 87, -16, 47, -111, -25, 2, 0, 2, 90, 0, 5, 105, 115, 75, 101, 121, 91, 0, 4, 100, 97, 116, 97, 116, 0, 2, 91, 66, 120, 114, 0, 40, 111, 114, 103, 46, 115, 112, 114, 105, 110, 103, 102, 114, 97, 109, 101, 119, 111, 114, 107, 46, 107, 97, 102, 107, 97, 46, 75, 97, 102, 107, 97, 69, 120, 99, 101, 112, 116, 105, 111, 110, 19, -40, -67, 105, 126, 123, 33, -76, 2, 0, 0, 120, 114, 0, 47, 111, 114, 103, 46, 115, 112, 114, 105, 110, 103, 102, 114, 97, 109, 101, 119, 111, 114, 107, 46, 99, 111, 114, 101, 46, 78, 101, 115, 116, 101, 100, 82, 117, 110, 116, 105, 109, 101, 69, 120, 99, 101, 112, 116, 105, 111, 110, 75, 126, 118, 72, -53, -113, -97, 0, 2, 0, 0, 120, 114, 0, 26, 106, 97, 118, 97, 46, 108, 97, 110, 103, 46, 82, 117, 110, 116, 105, 109, 101, 69, 120, 99, 101, 112, 116, 105, 111, 110, -98, 95, 6, 71, 10, 52, -125, -27, 2, 0, 0, 120, 114, 0, 19, 106, 97, 118, 97, 46, 108, 97, 110, 103, 46, 69, 120, 99, 101, 112, 116, 105, 111, 110, -48, -3, 31, 62, 26, 59, 28, -60, 2, 0, 0, 120, 114, 0, 19, 106, 97, 118, 97, 46, 108, 97, 110, 103, 46, 84, 104, 114, 111, 119, 97, 98, 108, 101, -43, -58, 53, 39, 57, 119, -72, -53, 3, 0, 4, 76, 0, 5, 99, 97, 117, 115, 101, 116, 0, 21, 76, 106, 97, 118, 97, 47, 108, 97, 110, 103, 47, 84, 104, 114, 111, 119, 97, 98, 108, 101, 59, 76, 0, 13, 100, 101, 116, 97, 105, 108, 77, 101, 115, 115, 97, 103, 101, 116, 0, 18, 76, 106, 97, 118, 97, 47, 108, 97, 110, 103, 47, 83, 116, 114, 105, 110, 103, 59, 91, 0, 10, 115, 116, 97, 99, 107, 84, 114, 97, 99, 101, 116, 0, 30, 91, 76, 106, 97, 118, 97, 47, 108, 97, 110, 103, 47, 83, 116, 97, 99, 107, 84, 114, 97, 99, 101, 69, 108, 101, 109, 101, 110, 116, 59, 76, 0, 20, 115, 117, 112, 112, 114, 101, 115, 115, 101, 100, 69, 120, 99, 101, 112, 116, 105, 111, 110, 115, 116, 0, 16, 76, 106, 97, 118, 97, 47, 117, 116, 105, 108, 47, 76, 105, 115, 116, 59, 120, 112, 115, 113, 0, 126, 0, 4, 113, 0, 126, 0, 12, 116, 0, 12, 65, 108, 119, 97, 121, 115, 32, 102, 97, 105, 108, 115, 117, 114, 0, 30, 91, 76, 106, 97, 118, 97, 46, 108, 97, 110, 103, 46, 83, 116, 97, 99, 107, 84, 114, 97, 99, 101, 69, 108, 101, 109, 101, 110, 116, 59, 2, 70, 42, 60, 60, -3, 34, 57, 2, 0, 0, 120, 112, 0, 0, 0, 18, 115, 114, 0, 27, 106, 97, 118, 97, 46, 108, 97, 110, 103, 46, 83, 116, 97, 99, 107, 84, 114, 97, 99, 101, 69, 108, 101, 109, 101, 110, 116, 97, 9, -59, -102, 38, 54, -35, -123, 2, 0, 4, 73, 0, 10, 108, 105, 110, 101, 78, 117, 109, 98, 101, 114, 76, 0, 14, 100, 101, 99, 108, 97, 114, 105, 110, 103, 67, 108, 97, 115, 115, 113, 0, 126, 0, 8, 76, 0, 8, 102, 105, 108, 101, 78, 97, 109, 101, 113, 0, 126, 0, 8, 76, 0, 10, 109, 101, 116, 104, 111, 100, 78, 97, 109, 101, 113, 0, 126, 0, 8, 120, 112, 0, 0, 0, 31, 116, 0, 34, 99, 111, 109, 46, 101, 120, 97, 109, 112, 108, 101, 46, 100, 101, 109, 111, 46, 77, 121, 66, 97, 100, 68, 101, 115, 101, 114, 105, 97, 108, 105, 122, 101, 114, 116, 0, 22, 77, 121, 66, 97, 100, 68, 101, 115, 101, 114, 105, 97, 108, 105, 122, 101, 114, 46, 106, 97, 118, 97, 116, 0, 11, 100, 101, 115, 101, 114, 105, 97, 108, 105, 122, 101, 115, 113, 0, 126, 0, 16, 0, 0, 0, 60, 116, 0, 50, 111, 114, 103, 46, 97, 112, 97, 99, 104, 101, 46, 107, 97, 102, 107, 97, 46, 99, 111, 109, 109, 111, 110, 46, 115, 101, 114, 105, 97, 108, 105, 122, 97, 116, 105, 111, 110, 46, 68, 101, 115, 101, 114, 105, 97, 108, 105, 122, 101, 114, 116, 0, 17, 68, 101, 115, 101, 114, 105, 97, 108, 105, 122, 101, 114, 46, 106, 97, 118, 97, 113, 0, 126, 0, 20, 115, 113, 0, 126, 0, 16, 0, 0, 0, -56, 116, 0, 71, 111, 114, 103, 46, 115, 112, 114, 105, 110, 103, 102, 114, 97, 109, 101, 119, 111, 114, 107, 46, 107, 97, 102, 107, 97, 46, 115, 117, 112, 112, 111, 114, 116, 46, 115, 101, 114, 105, 97, 108, 105, 122, 101, 114, 46, 69, 114, 114, 111, 114, 72, 97, 110, 100, 108, 105, 110, 103, 68, 101, 115, 101, 114, 105, 97, 108, 105, 122, 101, 114, 50, 116, 0, 31, 69, 114, 114, 111, 114, 72, 97, 110, 100, 108, 105, 110, 103, 68, 101, 115, 101, 114, 105, 97, 108, 105, 122, 101, 114, 50, 46, 106, 97, 118, 97, 113, 0, 126, 0, 20, 115, 113, 0, 126, 0, 16, 0, 0, 4, -12, 116, 0, 51, 111, 114, 103, 46, 97, 112, 97, 99, 104, 101, 46, 107, 97, 102, 107, 97, 46, 99, 108, 105, 101, 110, 116, 115, 46, 99, 111, 110, 115, 117, 109, 101, 114, 46, 105, 110, 116, 101, 114, 110, 97, 108, 115, 46, 70, 101, 116, 99, 104, 101, 114, 116, 0, 12, 70, 101, 116, 99, 104, 101, 114, 46, 106, 97, 118, 97, 116, 0, 11, 112, 97, 114, 115, 101, 82, 101, 99, 111, 114, 100, 115, 113, 0, 126, 0, 16, 0, 0, 0, 124, 113, 0, 126, 0, 28, 113, 0, 126, 0, 29, 116, 0, 11, 97, 99, 99, 101, 115, 115, 36, 51, 54, 48, 48, 115, 113, 0, 126, 0, 16, 0, 0, 5, -44, 116, 0, 68, 111, 114, 103, 46, 97, 112, 97, 99, 104, 101, 46, 107, 97, 102, 107, 97, 46, 99, 108, 105, 101, 110, 116, 115, 46, 99, 111, 110, 115, 117, 109, 101, 114, 46, 105, 110, 116, 101, 114, 110, 97, 108, 115, 46, 70, 101, 116, 99, 104, 101, 114, 36, 80, 97, 114, 116, 105, 116, 105, 111, 110, 82, 101, 99, 111, 114, 100, 115, 113, 0, 126, 0, 29, 116, 0, 12, 102, 101, 116, 99, 104, 82, 101, 99, 111, 114, 100, 115, 115, 113, 0, 126, 0, 16, 0, 0, 5, 52, 113, 0, 126, 0, 34, 113, 0, 126, 0, 29, 116, 0, 11, 97, 99, 99, 101, 115, 115, 36, 49, 54, 48, 48, 115, 113, 0, 126, 0, 16, 0, 0, 2, -123, 113, 0, 126, 0, 28, 113, 0, 126, 0, 29, 113, 0, 126, 0, 35, 115, 113, 0, 126, 0, 16, 0, 0, 2, 94, 113, 0, 126, 0, 28, 113, 0, 126, 0, 29, 116, 0, 14, 102, 101, 116, 99, 104, 101, 100, 82, 101, 99, 111, 114, 100, 115, 115, 113, 0, 126, 0, 16, 0, 0, 5, 14, 116, 0, 47, 111, 114, 103, 46, 97, 112, 97, 99, 104, 101, 46, 107, 97, 102, 107, 97, 46, 99, 108, 105, 101, 110, 116, 115, 46, 99, 111, 110, 115, 117, 109, 101, 114, 46, 75, 97, 102, 107, 97, 67, 111, 110, 115, 117, 109, 101, 114, 116, 0, 18, 75, 97, 102, 107, 97, 67, 111, 110, 115, 117, 109, 101, 114, 46, 106, 97, 118, 97, 116, 0, 14, 112, 111, 108, 108, 70, 111, 114, 70, 101, 116, 99, 104, 101, 115, 115, 113, 0, 126, 0, 16, 0, 0, 4, -55, 113, 0, 126, 0, 42, 113, 0, 126, 0, 43, 116, 0, 4, 112, 111, 108, 108, 115, 113, 0, 126, 0, 16, 0, 0, 4, -79, 113, 0, 126, 0, 42, 113, 0, 126, 0, 43, 113, 0, 126, 0, 46, 115, 113, 0, 126, 0, 16, 0, 0, 4, 10, 116, 0, 81, 111, 114, 103, 46, 115, 112, 114, 105, 110, 103, 102, 114, 97, 109, 101, 119, 111, 114, 107, 46, 107, 97, 102, 107, 97, 46, 108, 105, 115, 116, 101, 110, 101, 114, 46, 75, 97, 102, 107, 97, 77, 101, 115, 115, 97, 103, 101, 76, 105, 115, 116, 101, 110, 101, 114, 67, 111, 110, 116, 97, 105, 110, 101, 114, 36, 76, 105, 115, 116, 101, 110, 101, 114, 67, 111, 110, 115, 117, 109, 101, 114, 116, 0, 34, 75, 97, 102, 107, 97, 77, 101, 115, 115, 97, 103, 101, 76, 105, 115, 116, 101, 110, 101, 114, 67, 111, 110, 116, 97, 105, 110, 101, 114, 46, 106, 97, 118, 97, 116, 0, 6, 100, 111, 80, 111, 108, 108, 115, 113, 0, 126, 0, 16, 0, 0, 3, -34, 113, 0, 126, 0, 49, 113, 0, 126, 0, 50, 116, 0, 13, 112, 111, 108, 108, 65, 110, 100, 73, 110, 118, 111, 107, 101, 115, 113, 0, 126, 0, 16, 0, 0, 3, -97, 113, 0, 126, 0, 49, 113, 0, 126, 0, 50, 116, 0, 3, 114, 117, 110, 115, 113, 0, 126, 0, 16, 0, 0, 1, -1, 116, 0, 46, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 99, 111, 110, 99, 117, 114, 114, 101, 110, 116, 46, 69, 120, 101, 99, 117, 116, 111, 114, 115, 36, 82, 117, 110, 110, 97, 98, 108, 101, 65, 100, 97, 112, 116, 101, 114, 116, 0, 14, 69, 120, 101, 99, 117, 116, 111, 114, 115, 46, 106, 97, 118, 97, 116, 0, 4, 99, 97, 108, 108, 115, 113, 0, 126, 0, 16, 0, 0, 1, 10, 116, 0, 31, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 99, 111, 110, 99, 117, 114, 114, 101, 110, 116, 46, 70, 117, 116, 117, 114, 101, 84, 97, 115, 107, 116, 0, 15, 70, 117, 116, 117, 114, 101, 84, 97, 115, 107, 46, 106, 97, 118, 97, 113, 0, 126, 0, 55, 115, 113, 0, 126, 0, 16, 0, 0, 2, -20, 116, 0, 16, 106, 97, 118, 97, 46, 108, 97, 110, 103, 46, 84, 104, 114, 101, 97, 100, 116, 0, 11, 84, 104, 114, 101, 97, 100, 46, 106, 97, 118, 97, 113, 0, 126, 0, 55, 115, 114, 0, 38, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 67, 111, 108, 108, 101, 99, 116, 105, 111, 110, 115, 36, 85, 110, 109, 111, 100, 105, 102, 105, 97, 98, 108, 101, 76, 105, 115, 116, -4, 15, 37, 49, -75, -20, -114, 16, 2, 0, 1, 76, 0, 4, 108, 105, 115, 116, 113, 0, 126, 0, 10, 120, 114, 0, 44, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 67, 111, 108, 108, 101, 99, 116, 105, 111, 110, 115, 36, 85, 110, 109, 111, 100, 105, 102, 105, 97, 98, 108, 101, 67, 111, 108, 108, 101, 99, 116, 105, 111, 110, 25, 66, 0, -128, -53, 94, -9, 30, 2, 0, 1, 76, 0, 1, 99, 116, 0, 22, 76, 106, 97, 118, 97, 47, 117, 116, 105, 108, 47, 67, 111, 108, 108, 101, 99, 116, 105, 111, 110, 59, 120, 112, 115, 114, 0, 19, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 65, 114, 114, 97, 121, 76, 105, 115, 116, 120, -127, -46, 29, -103, -57, 97, -99, 3, 0, 1, 73, 0, 4, 115, 105, 122, 101, 120, 112, 0, 0, 0, 0, 119, 4, 0, 0, 0, 0, 120, 113, 0, 126, 0, 71, 120, 116, 0, 21, 102, 97, 105, 108, 101, 100, 32, 116, 111, 32, 100, 101, 115, 101, 114, 105, 97, 108, 105, 122, 101, 117, 113, 0, 126, 0, 14, 0, 0, 0, 17, 115, 113, 0, 126, 0, 16, 0, 0, 0, -28, 113, 0, 126, 0, 25, 113, 0, 126, 0, 26, 116, 0, 24, 100, 101, 115, 101, 114, 105, 97, 108, 105, 122, 97, 116, 105, 111, 110, 69, 120, 99, 101, 112, 116, 105, 111, 110, 115, 113, 0, 126, 0, 16, 0, 0, 0, -53, 113, 0, 126, 0, 25, 113, 0, 126, 0, 26, 113, 0, 126, 0, 20, 115, 113, 0, 126, 0, 16, 0, 0, 4, -12, 113, 0, 126, 0, 28, 113, 0, 126, 0, 29, 113, 0, 126, 0, 30, 115, 113, 0, 126, 0, 16, 0, 0, 0, 124, 113, 0, 126, 0, 28, 113, 0, 126, 0, 29, 113, 0, 126, 0, 32, 115, 113, 0, 126, 0, 16, 0, 0, 5, -44, 113, 0, 126, 0, 34, 113, 0, 126, 0, 29, 113, 0, 126, 0, 35, 115, 113, 0, 126, 0, 16, 0, 0, 5, 52, 113, 0, 126, 0, 34, 113, 0, 126, 0, 29, 113, 0, 126, 0, 37, 115, 113, 0, 126, 0, 16, 0, 0, 2, -123, 113, 0, 126, 0, 28, 113, 0, 126, 0, 29, 113, 0, 126, 0, 35, 115, 113, 0, 126, 0, 16, 0, 0, 2, 94, 113, 0, 126, 0, 28, 113, 0, 126, 0, 29, 113, 0, 126, 0, 40, 115, 113, 0, 126, 0, 16, 0, 0, 5, 14, 113, 0, 126, 0, 42, 113, 0, 126, 0, 43, 113, 0, 126, 0, 44, 115, 113, 0, 126, 0, 16, 0, 0, 4, -55, 113, 0, 126, 0, 42, 113, 0, 126, 0, 43, 113, 0, 126, 0, 46, 115, 113, 0, 126, 0, 16, 0, 0, 4, -79, 113, 0, 126, 0, 42, 113, 0, 126, 0, 43, 113, 0, 126, 0, 46, 115, 113, 0, 126, 0, 16, 0, 0, 4, 10, 113, 0, 126, 0, 49, 113, 0, 126, 0, 50, 113, 0, 126, 0, 51, 115, 113, 0, 126, 0, 16, 0, 0, 3, -34, 113, 0, 126, 0, 49, 113, 0, 126, 0, 50, 113, 0, 126, 0, 53, 115, 113, 0, 126, 0, 16, 0, 0, 3, -97, 113, 0, 126, 0, 49, 113, 0, 126, 0, 50, 113, 0, 126, 0, 55, 115, 113, 0, 126, 0, 16, 0, 0, 1, -1, 113, 0, 126, 0, 57, 113, 0, 126, 0, 58, 113, 0, 126, 0, 59, 115, 113, 0, 126, 0, 16, 0, 0, 1, 10, 113, 0, 126, 0, 61, 113, 0, 126, 0, 62, 113, 0, 126, 0, 55, 115, 113, 0, 126, 0, 16, 0, 0, 2, -20, 113, 0, 126, 0, 64, 113, 0, 126, 0, 65, 113, 0, 126, 0, 55, 113, 0, 126, 0, 69, 120, 0, 117, 114, 0, 2, 91, 66, -84, -13, 23, -8, 6, 8, 84, -32, 2, 0, 0, 120, 112, 0, 0, 0, 3, 102, 111, 111])], isReadOnly = false), key = null, value = null)
org.springframework.kafka.listener.ListenerExecutionFailedException: Listener failed; nested exception is org.springframework.kafka.support.serializer.DeserializationException: failed to deserialize; nested exception is java.lang.RuntimeException: Always fails
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.decorateException(KafkaMessageListenerContainer.java:1777) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeErrorHandler(KafkaMessageListenerContainer.java:1766) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:1679) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:1605) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:1510) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:1257) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:1007) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:927) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_212]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: org.springframework.kafka.support.serializer.DeserializationException: failed to deserialize; nested exception is java.lang.RuntimeException: Always fails
at org.springframework.kafka.support.serializer.ErrorHandlingDeserializer2.deserializationException(ErrorHandlingDeserializer2.java:228) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.support.serializer.ErrorHandlingDeserializer2.deserialize(ErrorHandlingDeserializer2.java:203) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.apache.kafka.clients.consumer.internals.Fetcher.parseRecord(Fetcher.java:1268) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.consumer.internals.Fetcher.access$3600(Fetcher.java:124) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.fetchRecords(Fetcher.java:1492) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.access$1600(Fetcher.java:1332) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:645) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:606) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1294) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1225) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1201) ~[kafka-clients-2.3.1.jar:na]
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doPoll(KafkaMessageListenerContainer.java:1034) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:990) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
... 4 common frames omitted
Caused by: java.lang.RuntimeException: Always fails
at com.example.demo.MyBadDeserializer.deserialize(MyBadDeserializer.java:31) ~[classes/:na]
at org.apache.kafka.common.serialization.Deserializer.deserialize(Deserializer.java:60) ~[kafka-clients-2.3.1.jar:na]
at org.springframework.kafka.support.serializer.ErrorHandlingDeserializer2.deserialize(ErrorHandlingDeserializer2.java:200) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
... 15 common frames omitted
我有一个使用Spring Cloud Stream和Spring Kafka的应用程序,它处理Avro消息。该应用程序运行良好,但现在我想添加一些错误处理。 目标是:捕获反序列化异常,使用异常详细信息原始Kafka消息自定义上下文信息构建新对象,并将此对象推送到专用Kafka主题。基本上是DLQ,但原始消息将被截获并修饰。 问题是:虽然我可以拦截异常,但我不知道如何从Kafka那里获取原始消息(下
使用Dropwizard框架构建API,我使用Jackson ObjectMapper遇到了这个反序列化问题。我同时使用Joda Time和Joda Money。对于JodaTime,定义JodaModule足以解决反序列化问题。但对于JodaMoney来说,JodaModule不足以解决反序列化问题(如果我错了,请纠正我)。因此,我为JodaTime保留了JodaModule,创建了JodaMo
有没有一种方法让我忽略这些异常并在消费者处移动偏移量?我想,因为我使用手动偏移提交,我有这个问题。有人知道如何配置kafka-avro-serializer-6.0.0.jar来完成我想要的任务吗? 多谢了。
我是Avro和Kafka的新手,我花了几天时间来发送关于Kafka主题的序列化数据...不成功。 让我来解释一下我想要达到的目标: 在生产者方面,我通过SOAP接收数据并发送关于Kafka主题的内容。我正在使用CXF从WSDL生成POJO,并且编写了相应的模式。我正在尝试做的是序列化由CXF解封的对象,并在我的Kafka主题上发送它们。 在web上找到的大多数示例中,Avro记录都是使用已知的模式
我无法使用功能供应商发送Avro消息。SCSt尝试将消息作为JSON发送,但失败。有人能指出是否需要任何其他配置吗? 这是供应商的功能bean 和配置