spring-cloud-bus | Spring Cloud event bus | Microservice library

My question is how to manage the multi instance with Spring Cloud Stream Kafka.

Let me explain, in a Spring Cloud Stream Microservices context (eureka, configserver, kafka) I want to have 2 instances of the same microservice. When I change a configuration in my GIT Repository, the configserver (via a webhook) will push a message into the Kafka topic.

If i use the same group-id in my microservice, only one of two instances will received the notification, and reload his spring context. But I need to refresh all instances ...

So, to do that, I have configured an unique group-id : ${spring.application.name}.bus.${hostname} It's work well, but the problem is, each time I start a new instance of my service, it create a new consumer group in kafka. Now i have a lot of unused consumer group.

[![consumers for a microservice][1]][1] [1]: https://i.stack.imgur.com/6jIzx.png

Here is the Spring Cloud Stream configuration of my service :

When the application stops, kinesis binder tries to unlock dynamoDB and throws unlock failed exception.

I followed this original post with the similar issue and updated spring-integration-aws version to v2.3.1.RELEASE. But still seeing the same error on application shut down.

I am trying to migrate a simple example code with Spring Cloud Config Server and RabbitMQ as Spring Cloud Bus (based on Spring Boot 1.5.22.RELEASE and Spring Cloud Brixton.SR7) to Spring Boot 2.2.6.RELEASE and Spring Cloud Hoxton.SR3. The example consists of a Config Server, a Config Client, GitLab as SCM and RabbitMQ(3.8 - Erlang 22.1.5). The code is compiling, starting up, the push webhook is triggered and can also be seen in the server's and client's log.

The problem is that the property updated in Git is not updated in the client. On the base of the Spring Boot 1.5.22.RELEASE and Spring Cloud Brixton.SR7 it works reliable. But if I do curl -X POST http://localhost:8889/actuator/bus-refresh manually, the property will be updated.

What can be the problem or which property have I forgotten to configure?

Here is my configuration/code:

GitLab (started as Docker container) Push WebHook: http://user:password@localhost:8889/monitor

RabbitMQ (started as Docker container) no particular configuration

pom.xml Root module of Config server and client:

I'm trying to use Spring cloud bus with Kafka in my microservices application, and indeed I could use it, but only data which is controlled by Spring cloud config server got refreshed!

I'm using jdbc back-end with my config server, and in order to simulate my need, I'm changing some value in properties file in one of my services, beside the properties table, and call the /monintor end point again (mentioned here section 4.3 https://www.baeldung.com/spring-cloud-bus); as a result, only data coming from properties table is changed.

This is the yml file for my Config server

In the documentation of Spring Cloud Bus (https://github.com/spring-cloud/spring-cloud-bus) it was mentioned like

The Bus starters cover Rabbit and Kafka, because those are the two most common implementations, but Spring Cloud Stream is quite flexible and binder will work combined with spring-cloud-bus.

In my project we can not maintain an another infrastructure for Rabbit or Kafka hence I want to use spring-cloud-stream-binder-aws-kinesis (https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis) with spring-cloud-bus. Can anyone guide me how can I do that??

I am learning spring cloud with consul as service discovery implementation, I followed a tutorial on internet. I am using eclipse and maven. The pom file is generated by Spring Initializr. From spring official documentation, they says "@EnableDiscoveryClient" is no more needed, so I commented it, but with this annoatation makes no difference.

my code is below:

We experienced the following scenario :

• We have a Kafka cluster composed of 3 nodes, each topic created has 3 partitions
• A message is sent through MessageChannel.send(), producing a record for, let's say, partition 1
• The broker acting as the partition leader for that partition fails

By default, MessageChannel.send() returns true and doesn't throw any exception, even if, eventually, the KafkaProducer can't send successfully the message. We observe, about 30 seconds after this call, the following message in the logs : Expiring 10 record(s) for helloworld-topic-1 due to 30008 ms has passed since batch creation plus linger time

In our case, this is not acceptable as we have to be sure that all messages are eventually delivered to Kafka, at the moment of the return of the call to MessageChannel.send().

We turned on spring.cloud.stream.kafka.bindings..producer.sync to true which does exactly as the documentation describes. It blocks the caller for the producer's acknowledgment of the success or the failure of the delivery (MessageTimeoutException, InterruptedException, ExecutionException), all of this controlled by KafkaProducerMessageHandler. It seems to be the best approach for us as the performance impact is negligible in our case.

But, do we need to take care of the retry ourselves if an exception is thrown ? (in our client code with @Retryable for instance)

Here is a simple project to experiment : https://github.com/phdezann/spring-cloud-bus-kafka-helloworld

I am using the Spring Cloud Config Server and able to detect the changes from the git repository and pass it to the Config clients.

There are two ways, I have implemented it:

1. After making changes (commit and push) in the git repository, I make a curl request curl -X POST http://server:port/bus/refresh and it works fine. For this I am using RabbitMQ as the Spring Cloud Bus.

Reference: http://tech.asimio.net/2017/02/02/Refreshable-Configuration-using-Spring-Cloud-Config-Server-Spring-Cloud-Bus-RabbitMQ-and-Git.html

1. After making changes (commit and push) in the git repository, I make a curl request curl -X POST http://server:port/refresh (with no /bus in the url) and it works fine. I am NOT using Spring Cloud Bus here.

Reference: https://spring.io/guides/gs/centralized-configuration/

So both works fine, so Is there any advantage of using Spring Cloud Bus, or in Production environment, will there be any issue with going without Spring Cloud Bus? As there will be extra effort needed to get setup the RabbitMQ Cluster (HA) as Spring Cloud Bus in production.

Thanks, David