spring-integration-kafka | Spring Integration for Apache Kafka | Pub Sub library

During a planned downtime for our Kafka cluster, we basically encountered the following issue How to specify timeout for sending message to RabbitMQ using Spring Cloud Stream? (with Kafka rather than RabbitMQ, obviously).

The answer from @GaryRussell:

The channel sendTimeout only applies if the channel itself can block, e.g. a QueueChannel with a bounded queue that is currently full; the caller will block until either space becomes available in the queue, or the timeout occurs.

In this case, the block is downstream of the channel so the sendTimeout is irrelevant (in any case, it's a DirectChannel which can't block anyway, the subscribed handler is called directly on the calling thread).

The actual blocking you are seeing is most likely in the socket.write() in the rabbitmq client, which does not have a timeout and is not interruptible; there is nothing that can be done by the calling thread to "time out" the write.

The only possible solution I am aware of is to force close the rabbit connection by calling resetConnection() on the connection factory.

explains quite well why the method in question (org.springframework.integration.channel.AbstractSubscribableChannel#doSend) does not take the timeout into account. However, this still seems a bit odd to me.

In spring-integration-kafka-3.2.1.RELEASE-sources.jar!/org/springframework/integration/kafka/outbound/KafkaProducerMessageHandler.java:566, we can see that, if sync behaviour is desired:

I am getting the following issues when trying to get Message Driven Channel Adaptor working with Spring-Kafka 2.3+. Does anyone have any example code which would help me?

1. org.springframework.kafka.listener.config.ContainerProperties does not actually exist.

2. org.springframework.kafka.listener.ContainerProperties does exist but produces the below issue when trying to run.

Description:

An attempt was made to call a method that does not exist. The attempt was made from the following location:

I am trying to write an app to consume messages from Kafka topic and then enrich the inbound message to add new headers. The versions in use are:

I am trying to setup a kafka consumer on legacy web application using xml configuration that is running Spring 5.2 release project (It is not a spring boot project). After looking up I found a project that sets up the kafka consumer using xml configuration like in here

https://docs.spring.io/spring-kafka/docs/1.3.11.RELEASE/reference/html/_spring_integration.html

However this does not give details on how to connect this with java in a spring web application . All examples are for spring-boot project. I did find out what configuring I need to add to xml

https://github.com/spring-projects/spring-integration-kafka

I also did find an example of Spring web application and kafka but it is for 4.X release and is from 2015.

https://techannotation.wordpress.com/2015/10/26/introduction-to-apache-kafka-using-spring/

Any help on any newer documentation or on how to setup a consumer in java spring web project with xml configuration in 2020 would be appreciated

I am wondering if there is a way to enable Stateful RetryTemplate using spring-cloud-stream-binder-kafka.

I noticed that there is a constructor

I am developing a sample Spring integration project which has kafka listener using spring integration. This is working fine with lower version of boot (boot - 1.5.9, kafka-client 1.0.0, spring-integration-kafka 2.3.0, spring-kafka 1.3.2). But getting below error while upgrading boot version from 1.5.9 to 2.1.x :-

Getting below error when i tried to use channel message store :- org.springframework.messaging.MessageDeliveryException: Failed to send message to channel 'executionFilterChannel' within timeout: -1