kandi background
Explore Kits

spring-kafka | Provides Familiar Spring Abstractions for Apache Kafka | Pub Sub library

 by   spring-projects Java Version: 3.0.1 License: Apache-2.0

 by   spring-projects Java Version: 3.0.1 License: Apache-2.0

kandi X-RAY | spring-kafka Summary

spring-kafka is a Java library typically used in Messaging, Pub Sub, Kafka applications. spring-kafka has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has high support. You can download it from GitHub, Maven.
[Join the chat at
Support
Support
Quality
Quality
Security
Security
License
License
Reuse
Reuse

kandi-support Support

  • spring-kafka has a highly active ecosystem.
  • It has 1825 star(s) with 1336 fork(s). There are 159 watchers for this library.
  • There were 9 major release(s) in the last 6 months.
  • There are 34 open issues and 1196 have been closed. On average issues are closed in 3 days. There are 6 open pull requests and 0 closed requests.
  • It has a positive sentiment in the developer community.
  • The latest version of spring-kafka is 3.0.1
spring-kafka Support
Best in #Pub Sub
Average in #Pub Sub
spring-kafka Support
Best in #Pub Sub
Average in #Pub Sub

quality kandi Quality

  • spring-kafka has 0 bugs and 0 code smells.
spring-kafka Quality
Best in #Pub Sub
Average in #Pub Sub
spring-kafka Quality
Best in #Pub Sub
Average in #Pub Sub

securitySecurity

  • spring-kafka has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
  • spring-kafka code analysis shows 0 unresolved vulnerabilities.
  • There are 0 security hotspots that need review.
spring-kafka Security
Best in #Pub Sub
Average in #Pub Sub
spring-kafka Security
Best in #Pub Sub
Average in #Pub Sub

license License

  • spring-kafka is licensed under the Apache-2.0 License. This license is Permissive.
  • Permissive licenses have the least restrictions, and you can use them in most projects.
spring-kafka License
Best in #Pub Sub
Average in #Pub Sub
spring-kafka License
Best in #Pub Sub
Average in #Pub Sub

buildReuse

  • spring-kafka releases are available to install and integrate.
  • Deployable package is available in Maven.
  • Build file is available. You can build the component from source.
  • Installation instructions are not available. Examples and code snippets are available.
  • It has 55422 lines of code, 4447 functions and 510 files.
  • It has medium code complexity. Code complexity directly impacts maintainability of the code.
spring-kafka Reuse
Best in #Pub Sub
Average in #Pub Sub
spring-kafka Reuse
Best in #Pub Sub
Average in #Pub Sub
Top functions reviewed by kandi - BETA

kandi has reviewed spring-kafka and discovered the below as its top functions. This is intended to give you an instant insight into spring-kafka implemented functionality, and help decide if they suit your requirements.

  • Determine the type of the given method .
    • Run the consumer .
      • Produce a Kafka message .
        • Retry batches .
          • Process the retry topic annotation .
            • Check topics config .
              • Build callback .
                • Initialize topics .
                  • Find all annotated listeners .
                    • Recursively traverse a list of records .

                      Get all kandi verified functions for this library.

                      Get all kandi verified functions for this library.

                      spring-kafka Key Features

                      Get involved with the Spring community on the Spring Community Forums. Please help out on the [StackOverflow](https://stackoverflow.com/questions/tagged/spring-kafka) by responding to questions and joining the debate.

                      Create [GitHub issues](https://github.com/spring-projects/spring-kafka/issues) for bugs and new features and comment and vote on the ones that you are interested in.

                      Github is for social coding: if you want to write code, we encourage contributions through pull requests from [forks of this repository](https://help.github.com/forking/). If you want to contribute code this way, please reference a GitHub issue as well covering the specific issue you are addressing.

                      Watch for upcoming articles on Spring by [subscribing](https://www.springsource.org/node/feed) to springframework.org

                      Use the Spring Framework code format conventions (import eclipse-code-formatter.xml from the root of the project if you are using Eclipse).

                      Make sure all new .java files to have a simple Javadoc class comment with at least an @author tag identifying you, and preferably at least a paragraph on what the class is for.

                      Add the ASF license header comment to all new .java files (copy from existing files in the project)

                      Add yourself as an @author to the .java files that you modify substantially (more than cosmetic changes).

                      Add some Javadocs and, if you change the namespace, some XSD doc elements.

                      A few unit tests would help a lot as well - someone has to do it.

                      If no-one else is using your branch, please rebase it against the current main (or another target branch in the main project).

                      spring-kafka Examples and Code Snippets

                      Community Discussions

                      Trending Discussions on spring-kafka
                      • Combining blocking and non-blocking retries in Spring Kafka
                      • EmbeddedKafka failing since Spring Boot 2.6.X : AccessDeniedException: ..\AppData\Local\Temp\spring.kafka*
                      • Kafka consumer not automatically reconnecting after outage
                      • How to avoid hitting all retryable topics for fatal-by-default exceptions?
                      • Spring Boot Logging to a File
                      • How to replace deprecated SeekToCurrentErrorHandler with DefaultErrorHandler (spring-kafka)?
                      • Correct Number of Partitions/Replicas for @RetryableTopic Retry Topics
                      • Spring Kafka configuration for high consumer processing time
                      • Destination resolver returned non-existent partition
                      • Dynamic destination in Spring Cloud Stream from Azure Event Hub to Kafka
                      Trending Discussions on spring-kafka

                      QUESTION

                      Combining blocking and non-blocking retries in Spring Kafka

                      Asked 2022-Apr-01 at 21:13

                      I am trying to implement non blocking rerties with single topic fixed back-off.

                      I am able to do so, thanks to documentation https://docs.spring.io/spring-kafka/reference/html/#single-topic-fixed-delay-retries.

                      Now I also need to perform a few blocked/local retries on main topic. I have been trying to implement this using DefaultErrorHandler as below:

                      @Bean
                      public DefaultErrorHandler retryErrorHandler() {
                              return new DefaultErrorHandler(new FixedBackOff(2000, 3));
                      }
                      

                      This does not seem to work with RetryableTopic.

                      I have also tried the following approach retry-topic-combine-blocking https://docs.spring.io/spring-kafka/reference/html/#retry-topic-combine-blocking using ListenerContainerFactoryConfigurer but issue I am facing here is creating beans KafkaConsumerBackoffManager, DeadLetterPublishingRecovererFactory and especially KafkaConsumerBackoffManager.

                      I need to know if this another way to achieve this using spring kafka framework or is there a way to construct above beans ?

                      ANSWER

                      Answered 2022-Apr-01 at 21:13

                      We're currently working on improving configuration for the non-blocking retries components.

                      For now, as documented here, you should inject these beans such as:

                      @Bean(name = RetryTopicInternalBeanNames.LISTENER_CONTAINER_FACTORY_CONFIGURER_NAME)
                      public ListenerContainerFactoryConfigurer lcfc(KafkaConsumerBackoffManager kafkaConsumerBackoffManager,
                                                                     DeadLetterPublishingRecovererFactory deadLetterPublishingRecovererFactory,
                                                                     @Qualifier(RetryTopicInternalBeanNames
                                                                             .INTERNAL_BACKOFF_CLOCK_BEAN_NAME) Clock clock) {
                          ListenerContainerFactoryConfigurer lcfc = new ListenerContainerFactoryConfigurer(kafkaConsumerBackoffManager, deadLetterPublishingRecovererFactory, clock);
                          lcfc.setBlockingRetryableExceptions(MyBlockingRetryException.class, MyOtherBlockingRetryException.class);
                          lcfc.setBlockingRetriesBackOff(new FixedBackOff(500, 5)); // Optional
                          return lcfc;
                      }}
                      

                      Also, there's a known issue where if you try to inject the beans before the first @KafkaListener bean with retryable topic is processed, the feature's component's beans won't be present in the context yet and will throw an error.

                      Does that happen to you?

                      We're currently working on a fix for this, but we should be able to work around that if that's your problem.

                      EDIT: Since the problem is that components are not instantiated yet, the most guaranteed workaround is to provide the components yourself.

                      Here's a sample on how to do that. Of course, adjust it accordingly if you need any further customization.

                          @Configuration
                          public static class SO71705876Configuration {
                      
                              @Bean(name = RetryTopicInternalBeanNames.LISTENER_CONTAINER_FACTORY_CONFIGURER_NAME)
                              public ListenerContainerFactoryConfigurer lcfc(KafkaConsumerBackoffManager kafkaConsumerBackoffManager,
                                      DeadLetterPublishingRecovererFactory deadLetterPublishingRecovererFactory) {
                                  ListenerContainerFactoryConfigurer lcfc = new ListenerContainerFactoryConfigurer(kafkaConsumerBackoffManager, deadLetterPublishingRecovererFactory, Clock.systemUTC());
                                  lcfc.setBlockingRetryableExceptions(IllegalArgumentException.class, IllegalStateException.class);
                                  lcfc.setBlockingRetriesBackOff(new FixedBackOff(500, 5)); // Optional
                                  return lcfc;
                              }
                      
                              @Bean(name = RetryTopicInternalBeanNames.KAFKA_CONSUMER_BACKOFF_MANAGER)
                              public KafkaConsumerBackoffManager backOffManager(ApplicationContext context) {
                                  PartitionPausingBackOffManagerFactory managerFactory =
                                          new PartitionPausingBackOffManagerFactory();
                                  managerFactory.setApplicationContext(context);
                                  return managerFactory.create();
                              }
                      
                              @Bean(name = RetryTopicInternalBeanNames.DEAD_LETTER_PUBLISHING_RECOVERER_FACTORY_BEAN_NAME)
                              public DeadLetterPublishingRecovererFactory dlprFactory(DestinationTopicResolver resolver) {
                                  return new DeadLetterPublishingRecovererFactory(resolver);
                              }
                      
                              @Bean(name = RetryTopicInternalBeanNames.DESTINATION_TOPIC_CONTAINER_NAME)
                              public DestinationTopicResolver destinationTopicResolver(ApplicationContext context) {
                                  return new DefaultDestinationTopicResolver(Clock.systemUTC(), context);
                              }
                      

                      In the next release this should not be a problem anymore. Please let me know if that works for you, or if any further adjustment to this workaround is necessary.

                      Thanks.

                      Source https://stackoverflow.com/questions/71705876

                      Community Discussions, Code Snippets contain sources that include Stack Exchange Network

                      Vulnerabilities

                      No vulnerabilities reported

                      Install spring-kafka

                      You can download it from GitHub, Maven.
                      You can use spring-kafka like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the spring-kafka component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

                      Support

                      For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

                      Find more information at:

                      Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
                      over 650 million Knowledge Items
                      Find more libraries
                      Reuse Solution Kits and Libraries Curated by Popular Use Cases
                      Explore Kits

                      Save this library and start creating your kit

                      Install
                      Clone
                      • https://github.com/spring-projects/spring-kafka.git

                      • gh repo clone spring-projects/spring-kafka

                      • git@github.com:spring-projects/spring-kafka.git

                      Share this Page

                      share link

                      See Similar Libraries in

                      Consider Popular Pub Sub Libraries
                      Try Top Libraries by spring-projects
                      Compare Pub Sub Libraries with Highest Support
                      Compare Pub Sub Libraries with Highest Quality
                      Compare Pub Sub Libraries with Highest Security
                      Compare Pub Sub Libraries with Permissive License
                      Compare Pub Sub Libraries with Highest Reuse
                      Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
                      over 650 million Knowledge Items
                      Find more libraries
                      Reuse Solution Kits and Libraries Curated by Popular Use Cases
                      Explore Kits

                      Save this library and start creating your kit