spring-boot-kafka | Simple Spring Boot demo project with Apache Kafka | Pub Sub library

I've written a basic spring boot service that consumes some data via rest API and publishes it to rabbitmq and kafka.

To test the service class handling kafka producing, I followed this guide: https://www.baeldung.com/spring-boot-kafka-testing

In isolation, the test (KafkaMessagingServiceImplTest) works perfectly both in intellij idea and via mvn on the command line. Running all project tests in idea works fine. However, when I run all project tests via maven on the command line, this test fails with an NPE when trying to make the assertion on the payload String.

I've narrowed down the location of the root problem to another test class (AppPropertiesTest) which is solely testing my AppProperties component (which is a component I use to pull config from application.properties in a tidy way). When, and only when, the tests within that test class are run alongside the failing test using 'mvn clean install' in project root, does the NPE show up. Commenting out the tests in this class or annotating it with @DirtiesContext fixes the problem. Apparently something loaded into the spring context by this test class causes an issue with the timing/order of events/countdownlatch in the other test. Of course, I don't want to use @DirtiesContext as it can lead to a much slower build as the project increases in complexity. It also does not explain the problem.. and I can't handle that :)

AppPropertiesTest uses constructor injection to inject the AppProperties component. It also extends a abstract class 'GenericServiceTest' which is annotated by:

I have implemented a simple application for streaming logs to apache kafka.In the producer, I have implemented log4j2.xml file for the configuration and now I have to convert .xml file into properties file. It is working properly with .xml but when I use properties file then following error has occurred.

I am facing an issue in my spring-boot(2.1.7.RELEASE) app server which uses spring-boot-Kafka('spring-Kafka' -'2.2.7.RELEASE') integration library to access my Kafka topics.

When my Kafka brokers are down, my application is unable to start.

This is what I get:

I'm trying to commit a message just after reading it from the topic. I've followed this link (https://www.confluent.io/blog/apache-kafka-spring-boot-application) to create a Kafka consumer with spring. Normally it works perfect and the consumer gets the message and waits till anotherone enters in the queue. But the problem is that when I process this messages it takes a lot of time (circa 10 minutes) the kafka queue thinks that the message is not consumed (commited) and the consumers reads it again and again. I have to say that when my process time is less than 5 minutes it works well but when it lastas longer it doesn't commit the message.

I've looked for some answers around but it doesn't help me because I'm not using the same source code (and of course a different structure). I've tried to send asynchronous methods and also to commit asynchronously the message but I've failed. Some of the sources are:

Spring Boot Kafka: Commit cannot be completed since the group has already rebalanced

https://www.confluent.io/blog/tutorial-getting-started-with-the-new-apache-kafka-0-9-consumer-client/

https://dzone.com/articles/kafka-clients-at-most-once-at-least-once-exactly-o

Kafka 0.10 Java consumer not reading message from topic

https://github.com/confluentinc/confluent-kafka-dotnet/issues/470

The main class is here :

I am writing a demo application to create a Kafka Producer. I created a topic and ran a producer and consumer on Kafka, it seems to be working. I am writing a spring application to create a producer.I'm passing the name as part of the call. When I go to "http://localhost:8080/kafka/publish/Peter", I'm getting a whitelabel error "Failed to construct kafka producer". Please help.

The main application is: SpringBootKafkaProducerApplication.java

pom.xml