kafka-connect-transform-common | Common Transforms for Kafka Connect | Pub Sub library

by jcustenborder Java Version: 0.1.0.14 License: Apache-2.0

X-Ray Key Features Code Snippets Community Discussions(1)Vulnerabilities Install Support

kandi X-RAY | kafka-connect-transform-common Summary

kafka-connect-transform-common is a Java library typically used in Messaging, Pub Sub, Kafka applications. kafka-connect-transform-common has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has low support. You can download it from GitHub, Maven.

Documentation | Confluent Hub. This project contains common transformations for every day use cases with Kafka Connect.

Support

Quality

Security

License

Reuse

Support

kafka-connect-transform-common has a low active ecosystem.

It has 123 star(s) with 41 fork(s). There are 4 watchers for this library.

It had no major release in the last 12 months.

There are 26 open issues and 33 have been closed. On average issues are closed in 27 days. There are 4 open pull requests and 0 closed requests.

It has a neutral sentiment in the developer community.

The latest version of kafka-connect-transform-common is 0.1.0.14

Quality

kafka-connect-transform-common has 0 bugs and 149 code smells.

Security

kafka-connect-transform-common has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

kafka-connect-transform-common code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

kafka-connect-transform-common is licensed under the Apache-2.0 License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

kafka-connect-transform-common releases are available to install and integrate.

Deployable package is available in Maven.

Build file is available. You can build the component from source.

Installation instructions are not available. Examples and code snippets are available.

It has 4773 lines of code, 330 functions and 62 files.

It has low code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed kafka-connect-transform-common and discovered the below as its top functions. This is intended to give you an instant insight into kafka-connect-transform-common implemented functionality, and help decide if they suit your requirements.

Process the given schema and value
Process a map value
Transforms a schema into a timestamp value
Transforms input into output schema
Returns state for a given schema
Extract XPath and value
Helper method to extract a XPath string from a data object
Process inputSchema and outputSchema
Process input field
Build schema
Returns a list of supported header types
Process the input
Process map
Applies the debug information
Returns the output schema and the output schema
Process the inputSchema and return a struct
Create a schema for a record
Create the output schema for the given input record
Process the given input schema
Configures the output
Process the input schema
Process an input
Process an output field
Configures the converter

Get all kandi verified functions for this library.

kafka-connect-transform-common Key Features

No Key Features are available at this moment for kafka-connect-transform-common.

kafka-connect-transform-common Examples and Code Snippets

No Code Snippets are available at this moment for kafka-connect-transform-common.

Community Discussions

Trending Discussions on kafka-connect-transform-common

kafka-connect-elasticsearch: How to delete document based on certain value in header of the Kafka topic

QUESTION

kafka-connect-elasticsearch: How to delete document based on certain value in header of the Kafka topic

Asked 2021-Apr-21 at 01:45

I am getting up to speed here with Kafka Connect. Trying to use Kafka Connect Elasticsearch Service sink connector to move our data from Kafka to Elasticsearch. I have a processing stream that looks like this:

File record from s3->custom processing from source application which publishes to ->kafka topic->Kafka connect->Elastic Search

This works for the scenario of create/update. However we want to handle the delete scenario for the file. Our application publishes an event for delete action and sets that as part of the header value in the Kafka message. Instead of updating the document in elastic with this delete action info, we would like to delete the document itself.

How we can achieve this using Kafka Connect to read this header value and issue a delete of document for the given key from the Elastic?

Thanks for your help in advance.

Regards, Vikas

EDITED: Example of message I am trying to transform:

...

ANSWER

Answered 2021-Apr-21 at 01:45

I was able to achieve this. Ended up writing a custom SMT. Using the SMT I had access to the connect record including the header and the value. So I just read the header values on by one and when encountered the value I was interested in, I set the connect record's value to null. In addition to this Kafka Connect also exposes below parameter:

Source https://stackoverflow.com/questions/66977092

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install kafka-connect-transform-common

You can download it from GitHub, Maven.
You can use kafka-connect-transform-common like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the kafka-connect-transform-common component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

Support

Contributions are always welcomed! Before you start any development please create an issue and start a discussion. Create a pull request against your newly created issue and we're happy to see if we can merge your pull request. First and foremost any time you're adding code to the code base you need to include test coverage. Make sure that you run mvn clean package before submitting your pull to ensure that all of the tests, checkstyle rules, and the package can be successfully built.

Find more information at: