spring-cloud-stream-binder-aws-kinesis | Spring Cloud Stream binder for AWS Kinesis | Web Services library

by spring-cloud Java Version: v4.0.0 License: Apache-2.0

X-Ray Key Features Code Snippets Community Discussions(4)Vulnerabilities Install Support

kandi X-RAY | spring-cloud-stream-binder-aws-kinesis Summary

spring-cloud-stream-binder-aws-kinesis is a Java library typically used in Web Services applications. spring-cloud-stream-binder-aws-kinesis has no vulnerabilities, it has build file available, it has a Permissive License and it has low support. However spring-cloud-stream-binder-aws-kinesis has 2 bugs. You can download it from GitHub.

This repository contains the code for the Spring Cloud Stream Binder implementation for AWS Kinesis. See AWS Kinesis Binder Reference Manual for more information.

Support

Quality

Security

License

Reuse

Support

spring-cloud-stream-binder-aws-kinesis has a low active ecosystem.

It has 86 star(s) with 91 fork(s). There are 24 watchers for this library.

It had no major release in the last 12 months.

There are 0 open issues and 140 have been closed. On average issues are closed in 235 days. There are no pull requests.

It has a neutral sentiment in the developer community.

The latest version of spring-cloud-stream-binder-aws-kinesis is v4.0.0

Quality

spring-cloud-stream-binder-aws-kinesis has 2 bugs (1 blocker, 0 critical, 1 major, 0 minor) and 45 code smells.

Security

spring-cloud-stream-binder-aws-kinesis has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

spring-cloud-stream-binder-aws-kinesis code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

spring-cloud-stream-binder-aws-kinesis is licensed under the Apache-2.0 License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

spring-cloud-stream-binder-aws-kinesis releases are available to install and integrate.

Build file is available. You can build the component from source.

It has 6435 lines of code, 184 functions and 53 files.

It has medium code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed spring-cloud-stream-binder-aws-kinesis and discovered the below as its top functions. This is intended to give you an instant insight into spring-cloud-stream-binder-aws-kinesis implemented functionality, and help decide if they suit your requirements.

Creates a consumer endpoint
Return the kinesis client lib configuration
Creates the KCP consumer endpoint
Create the KinesisConsumer endpoint
Lists shards for a Stream
Build the fake next token
The DynamoDB store registry
Gets the locks
Create message handler
Creates the Kinesis message handler
Creates AmazonDynamoDB client
Convert the headers to a map
The KinesProducer bean
Creates a producer destination
Initialize the bean
Creates an Amazon CloudWatch async client
The default bindings prefix for this binding
Default AmazonDynamoDB streams
Returns the extended consumer properties for the given channel name
Returns the extended properties entry class
The kinesis - service bean
Returns the health of the Amazon Kinesis
The KinesisMessageBinder bean
Add partition key to output channel
Create a KinesisConsumer destination
The Amazon Kinesisis client bean

Get all kandi verified functions for this library.

spring-cloud-stream-binder-aws-kinesis Key Features

No Key Features are available at this moment for spring-cloud-stream-binder-aws-kinesis.

spring-cloud-stream-binder-aws-kinesis Examples and Code Snippets

No Code Snippets are available at this moment for spring-cloud-stream-binder-aws-kinesis.

Community Discussions

Trending Discussions on spring-cloud-stream-binder-aws-kinesis

How to disable CloudWatch metrics for KPL/KCL with Spring Cloud Stream

Spring Boot , Spring Integration RabbitMQ & AWS Kinesis Integration

Manual Acknowledgement (Checkpointing) of Messages: Spring Cloud Stream Kenesis Binder

AWS SDK 1.11.844 is throwing Continuously ERROR while connecting to Kinesis Stream

QUESTION

How to disable CloudWatch metrics for KPL/KCL with Spring Cloud Stream

Asked 2021-Dec-07 at 21:33

I am using the Spring Cloud Stream Binder for Kinesis with KPL/KCL enabled. We would like to disable Cloudwatch metrics without having to manage the configuration of KPL and KCL ourselves (completely overriding the beans). We would like to use the same bean definition for the KinesisProducerConfiguration and each of the KinesisClientLibConfiguration besides the KinesisProducerConfiguration.setMetricsLevel() and KinesisClientLibConfiguration.withMetricsLevel(...) properties.

For reference, here is where the AWS beans are defined in the Spring Cloud Stream Kinesis Binder: KinesisBinderConfiguration.java

What would be the most effective way to do this?

Any help is appreciated! Thanks.

...

ANSWER

Answered 2021-Dec-07 at 21:33

The framework does not provide any of the KinesisClientLibConfiguration. It is your project responsibility to expose such a bean and with whatever options you need: https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis/blob/main/spring-cloud-stream-binder-kinesis-docs/src/main/asciidoc/overview.adoc#kinesis-consumer-properties

Starting with version 2.0.1, beans of KinesisClientLibConfiguration type can be provided in the application context to have a full control over Kinesis Client Library configuration options.

The producer side indeed is covered by the KinesisProducerConfiguration bean in the KinesisBinderConfiguration:

Source https://stackoverflow.com/questions/70254662

QUESTION

Spring Boot , Spring Integration RabbitMQ & AWS Kinesis Integration

Asked 2021-Sep-16 at 16:50

I have a requirement to consume messages from rabbitMQ, do some processing and finally publish messages to the Kinesis Data stream. We are already using Spring Boot, Spring Integration Core & Spring Integration AQMP 5.5.1 Integration Flows for consuming messages from RabbitMQ. We are not using Spring Cloud Stream for any of our projects.

What spring library do you suggest for the use case to publish messages to the Kinesis data stream? After going through the Spring docs, I see a couple of options available. Could you please advise which is the best one to pursue?

spring-cloud-stream-binder-aws-kinesis
spring-integration-aws

...

ANSWER

Answered 2021-Sep-16 at 16:50

As long as you are not concerned about Spring Cloud Stream, you should not bring that AWS Kinesis Binder for Spring Cloud Stream dependency into your project. Just because it is not going to work without Spring Cloud Stream features in your project.

Since your application is really a Spring Integration one, you definitely need to use that Spring Integration for AWS dependency. It comes with a KinesisMessageHandler implementation to produce records into Kinesis stream.

See its docs for more info: https://github.com/spring-projects/spring-integration-aws#outbound-channel-adapter-3. Such a handler should be declared as a bean and can be used in the .handle() endpoint of an IntegrationFlow definition. See docs about existing handlers and missed Java DSL factories: https://docs.spring.io/spring-integration/docs/current/reference/html/dsl.html#java-dsl-protocol-adapters

UPDATE

How to use Kinesis MH in Java DSL:

Source https://stackoverflow.com/questions/69200319

QUESTION

Manual Acknowledgement (Checkpointing) of Messages: Spring Cloud Stream Kenesis Binder

Asked 2021-Mar-05 at 07:55

We are trying to port our Spring Cloud Stream application which consumes messages from Kafka to AWS Kenesis. We require manual acknowledgement for handling certain timeout conditions.

For Kafka we use property autocommitoffset to false and use the ACKNOWLEDGEMENT header to handle the manual acknowledgement.

I went through the documentation for Spring Cloud Stream went through the below: https://dataflow.spring.io/docs/recipes/kinesis/simple-producer-consumer/ https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis/blob/master/spring-cloud-stream-binder-kinesis-docs/src/main/asciidoc/overview.adoc

But could not find any solutions. Any pointers would be very helpful.

...

ANSWER

Answered 2021-Mar-05 at 07:52

After some more searching, found the solution :

In Kenesis the shard is equivalent to partition and checkpoint to offset

In Application Yml:

Source https://stackoverflow.com/questions/66485854

QUESTION

AWS SDK 1.11.844 is throwing Continuously ERROR while connecting to Kinesis Stream

Asked 2020-Sep-05 at 09:20

I am Using Spring Cloud Data Stream and Kinesis Spring Binder to connect to Kinesis: https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis/blob/master/spring-cloud-stream-binder-kinesis-docs/src/main/asciidoc/overview.adoc By default, it uses aws-sdk 1.11.415 and which can't be used for Assumed Role while deploying in Kubernetes environment (AWS EKS). I have figured out that I need to upgrade the library to aws-sdk 1.11.844 to support the WebIdentityToken based credential provider. This is working in Kubernetes. Now from my local machine, I am trying to connect to Kinesis by enabling instance profile (disabling the WebIdentityToken based flow), it keeps on throwing below error and not connecting. when I comment this upgraded library it goes back to aws-sdk 1.11.415 which is working as expected. But I need to upgrade this for WebIdentityToken based credential provider.

Please help me if anybody having any solution for this. I can't change to AWS SDKV2 as Spring Binder will not support that. Any other solution will be helpful.

...

ANSWER

Answered 2020-Sep-02 at 22:28

The problem is that your local machine doesn't have an instance profile.

If you look at the stack trace, you'll see EC2MetadataUtils.getItems(). This is trying to read from the instance metadata endpoint, http://169.254.169.254. On an EC2 instance that returns quickly; on your local machine it will time-out because that IP address doesn't exist.

If you're looking to consume AWS services from your local machine, you will have to use a credentials provider that retrieves the information locally. I recommend using DefaultAWSCredentialsProviderChain, which looks for local configuration (either in $HOME/.aws or via environment variables or system properties), and also supports instance profiles (in case you're running on an EC2 instance).

Source https://stackoverflow.com/questions/63711899

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install spring-cloud-stream-binder-aws-kinesis

You can download it from GitHub.
You can use spring-cloud-stream-binder-aws-kinesis like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the spring-cloud-stream-binder-aws-kinesis component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: