kafka-connect-oracle | Kafka Source Connector For Oracle | Pub Sub library

by erdemcer Java Version: Current License: Apache-2.0

X-Ray Key Features Code Snippets Community Discussions(2)Vulnerabilities Install Support

kandi X-RAY | kafka-connect-oracle Summary

kafka-connect-oracle is a Java library typically used in Messaging, Pub Sub, Kafka, Oracle applications. kafka-connect-oracle has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has low support. You can download it from GitHub.

kafka-connect-oracle is a Kafka source connector for capturing all row based DML changes from Oracle database and streaming these changes to Kafka. Change data capture logic is based on Oracle LogMiner solution. Only committed changes are pulled from Oracle which are Insert, Update, Delete operations. All streamed messages have related full "sql_redo" statement and parsed fields with values of sql statements. Parsed fields and values are kept in proper field type in schemas. Messages have old (before change) and new (after change) values of row fields for DML operations. Insert operation has only new values of row tagged as "data". Update operation has new data tagged as "data" and also contains old values of row before change tagged as "before". Delete operation only contains old data tagged as "before".

Support

Quality

Security

License

Reuse

Support

kafka-connect-oracle has a low active ecosystem.

It has 273 star(s) with 147 fork(s). There are 38 watchers for this library.

It had no major release in the last 6 months.

There are 60 open issues and 37 have been closed. On average issues are closed in 14 days. There are no pull requests.

It has a neutral sentiment in the developer community.

The latest version of kafka-connect-oracle is current.

Quality

kafka-connect-oracle has 0 bugs and 0 code smells.

Security

kafka-connect-oracle has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

kafka-connect-oracle code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

kafka-connect-oracle is licensed under the Apache-2.0 License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

kafka-connect-oracle releases are not available. You will need to build from source code and install.

Build file is available. You can build the component from source.

Installation instructions, examples and code snippets are available.

kafka-connect-oracle saves you 776 person hours of effort in developing the same functionality from scratch.

It has 1785 lines of code, 146 functions and 17 files.

It has high code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed kafka-connect-oracle and discovered the below as its top functions. This is intended to give you an instant insight into kafka-connect-oracle implemented functionality, and help decide if they suit your requirements.

Main thread
Search for log files
Calculate the offset for the source offsets
Create a SourceRecord from DML row
Polls from the kafka cluster
Load table
Parses the SQL
Create the data schema
Opens the Kafka connector session
Returns the database name
Returns the database version number
Shutdown the thread
Parse table white list
Get the table white list
Gets the log files
Execute a callable statement
Stop the driver
Get the current SCN value
Initialize the connector
Retrieves a list of strings for the specified task
Get configuration
Stops the database connection
Returns the version string
Get row data type
Get the schema for the table record
String representation of this transaction

Get all kandi verified functions for this library.

kafka-connect-oracle Key Features

No Key Features are available at this moment for kafka-connect-oracle.

kafka-connect-oracle Examples and Code Snippets

No Code Snippets are available at this moment for kafka-connect-oracle.

Community Discussions

Trending Discussions on kafka-connect-oracle

Creating pod and service for custom kafka connect image with kubernetes

Confluent Kafka Connect: New records are not populating my table-specific topic

QUESTION

Creating pod and service for custom kafka connect image with kubernetes

Asked 2022-Jan-26 at 16:23

I had successfully created a custom kafka connector image containing confluent hub connectors.

I am trying to create pod and service to launch it in GCP with kubernetes.

How should I configure yaml file ? The next part of code I took from quick-start guide. This is what I've tried: Dockerfile:

...

ANSWER

Answered 2022-Jan-26 at 16:23

After some retries I found out that I just had to wait a little bit longer.

Source https://stackoverflow.com/questions/70679072

QUESTION

Confluent Kafka Connect: New records are not populating my table-specific topic

Asked 2022-Jan-21 at 17:50

I have setup a simple Kafka connect process to connect to and detect changes in an Oracle CDB/PDB environment.

Have setup all components successfully with no errors - tables created, users can query, topics get created etc. However, I'm facing an issue with the CDC process where "New records are not populating my table-specific topic".

There is an entry for this issue in the confluent troubleshooting guide here: https://docs.confluent.io/kafka-connect-oracle-cdc/current/troubleshooting.html#new-records-are-not-populating-my-table-specific-topic

But when reading this I'm unsure as it can be interpreted multiple ways depending on how you look at it:

New records are not populating my table-specific topic
The existing schema (of the table-specific topic?) may not be compatible with the redo log topic (incompatible redo schema or incompatible redo topic itself?). Removing the schema (the table-specific or redo logic schema?) or using a different redo log topic may fix this issue (a different redo topic? why?)

From this I've had no luck trying to get my process to detect the changes. Looking for some support to fully understand this solution above from Confluent.

...

ANSWER

Answered 2022-Jan-21 at 17:50

In our case the reason was in absence of redo.log.consumer.bootstrap.servers setting. Also, the redo topic name setting redo.log.topic.name was important to set.

Assumption: it seems, that in case of 'snapshot' mode, the connector brings initial data to table topics and then starts to pull the redo log and write relevant entries to 'redo' topic. In parallel, as a separate task, it starts a consumer task to read from redo topic, and that consumer task actually writes CDC changes to table topics. That's why the 'redo.log.consumer.*' settings are relevant to configure.

Source https://stackoverflow.com/questions/70096601

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install kafka-connect-oracle

The database must be in archivelog mode and supplemental logging must be enabled.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: