spring-kafka | spring-kafka projects | Pub Sub library

spring-kafka creates a ValueSerializer instance in the AbstractConfig class using a no-args constructor.

I can see that JsonSerializer has an ObjectMapper constructor which I would like to use to inject a preconfigured ObjectMapper bean.

The default ObjectMapper includes null values in the response which I would like to remove. I added spring.jackson.default-property-inclusion: NON_EMPTY to my properties.yml but since Spring creates a default instance, this does not help me.

Could someone point me in the right direction?

I have been facing the exception below on the Kafka consumer side. Surprisingly, this issue is not consistent and an older version of the code (with the exact same configuration but some new unrelated features) works as expected. Could anyone help in determining what could be causing this?

A simple spring-boot-kafka which consumes from a topic on a network cluster:

Errors:

Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected

Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.

Puzzle:

The configured broker is not local, it's BROKER_1.FOO.NET:9094, and it is available.

pom.xml

How can I change the logging for Springboot Kafka? I'm seeing over 2M messages on our Splunk server and nothing is working:

I have a question about handling deserialization exceptions in Spring Cloud Stream while processing batches (i.e. batch-mode: true).

Per the documentation here, https://docs.spring.io/spring-kafka/docs/2.5.12.RELEASE/reference/html/#error-handling-deserializer, (looking at the implementation of FailedFooProvider), it looks like this function should return a subclass of the original message.

Is the intent here that a list of both Foo's and BadFoo's will end up at the original @StreamListener method, and then it will be up to the code (i.e. me) to sort them out and handle separately? I suspect this is the case, as I've read that the automated DLQ sending isn't desirable for batch error handling, as it would resubmit the whole batch.

And if this is the case, what if there is more than one message type received by the app via different @StreamListener's, say Foo's and Bar's. What type should the value function return in that case? Below is the pseudo code to illustrate the second question?

I want to stop polling for a specific topic at a specific time.

• spring-boot 2.X
• spring-kafka 2.5.5
• Kafka version 2.5.1

For example, even if a message comes in to the TEST topic partition, the message is piled up in the partition from 00 to 01, and there is no consumption.

After 01 o'clock, I want to consume the message again on the TEST topic.

How to pause and resume?

I'm using spring-kafka '2.2.7.RELEASE' to create a batch consumer and I'm trying to understand How can i configure a kafka batch consumer to retry a pre-defined no of times using SeekToCurrentBatchErrorHandler?

I see the one of the SeekToCurrentErrorHandler constructors takes 'maxFailures' as an argument but I don't see any such option for SeekToCurrentBatchErrorHandler. Please suggest.

I am developping a simple spring boot application using spring cloud stream and kafka.

I get this error when I added kafka consumer bean.

Spring boot version: 2.5.0

Spring cloud version: 2020.0.3-SNAPSHOT

Kafka client version: 2.7.1

Error log:

An attempt was made to call a method that does not exist. The attempt was made from the following location:

org.springframework.cloud.stream.binder.kafka.KafkaMessageChannelBinder.createConsumerEndpoint(KafkaMessageChannelBinder.java:716)

The following method did not exist:

org.springframework.kafka.listener.ContainerProperties.setAckOnError(Z)V

pom.xml file:

We have an existing application which is working fine with the SpringBoot 2.2.2.RELEASE. Now we tried to upgrade it to the SpringBoot 2.4.2 version and application is not getting started and throws the following error. In the classpath I could see only one spring-webmvc-5.3.2.jar file.

Below is the pom.xml for the referance: