kafka-connect-mq-source | repository contains a Kafka Connect source connector | Stream Processing library

by ibm-messaging Java Version: v1.3.2 License: Apache-2.0

X-Ray Key Features Code Snippets(3)Community Discussions(3)Vulnerabilities Install Support

kandi X-RAY | kafka-connect-mq-source Summary

kafka-connect-mq-source is a Java library typically used in Data Processing, Stream Processing, Kafka applications. kafka-connect-mq-source has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has low support. You can download it from GitHub.

This repository contains a Kafka Connect source connector for copying data from IBM MQ into Apache Kafka.

Support

Quality

Security

License

Reuse

Support

kafka-connect-mq-source has a low active ecosystem.

It has 86 star(s) with 79 fork(s). There are 24 watchers for this library.

It had no major release in the last 12 months.

There are 7 open issues and 56 have been closed. On average issues are closed in 180 days. There are 4 open pull requests and 0 closed requests.

It has a neutral sentiment in the developer community.

The latest version of kafka-connect-mq-source is v1.3.2

Quality

kafka-connect-mq-source has 0 bugs and 0 code smells.

Security

kafka-connect-mq-source has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

kafka-connect-mq-source code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

kafka-connect-mq-source is licensed under the Apache-2.0 License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

kafka-connect-mq-source releases are available to install and integrate.

Build file is available. You can build the component from source.

Installation instructions, examples and code snippets are available.

It has 1157 lines of code, 38 functions and 12 files.

It has medium code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed kafka-connect-mq-source and discovered the below as its top functions. This is intended to give you an instant insight into kafka-connect-mq-source implemented functionality, and help decide if they suit your requirements.

This method verifies that all messages in the MQ queue have been committed
Puts all messages to a given queue
Poll for records from the SourceTask
Configure connector
Verify the batch individual messages
Commit a record
Performs an authenticating queue manager
Puts all messages into queue
Builds a Kafka record from the JMS bytes message
Gets the key to use for a Kafka SourceRecord
Build from JMS map message
Generates a set of configurations based on the current configuration
Start the Connector
Builds a Kafka record from a JMS message
Builds a Kafka record from a JMS message
Builds a Kafka record from a JMS map message
Login JMQ CorrelationID with jqsource ID
Builds a Kafka message
Gets the value schema for a SourceRecord
Test to see if the destination destination is valid
Configure this class
Verify message id as key is expected
This test is used to validate JMS correlation ID bytes as expected by JMS source
Verify JMS messages
Verify JMS message headers
Configuration for this connector

Get all kandi verified functions for this library.

kafka-connect-mq-source Key Features

No Key Features are available at this moment for kafka-connect-mq-source.

kafka-connect-mq-source Examples and Code Snippets

Kafka Connect source connector for IBM MQ,Data formats

Java

Lines of Code : 7

License : Permissive (Apache-2.0)

Copy

value.converter=org.apache.kafka.connect.converters.ByteArrayConverter

mq.message.body.jms=true
value.converter=org.apache.kafka.connect.converters.StringConverter

mq.message.body.jms=true
value.converter=org.apache.kafka.connect.converters.ByteArr

Externalizing secrets

Java

Lines of Code : 6

License : Permissive (Apache-2.0)

Copy

secret-key=password

# Additional properties for the worker configuration to enable use of ConfigProviders
# multiple comma-separated provider types can be specified here
config.providers=file
config.providers.file.class=org.apache.kafka.common.confi

Kafka Connect source connector for IBM MQ,Running with Docker

Java

Lines of Code : 3

License : Permissive (Apache-2.0)

Copy

docker run -v $(pwd)/config:/opt/kafka/config -p 8083:8083 kafkaconnect-with-mq-source:1.3.0

curl -X POST -H "Content-Type: application/json" http://localhost:8083/connectors \
  --data "@./config/mq-source.json"

Community Discussions

Trending Discussions on kafka-connect-mq-source

userid is getting changed at the MQ side from kafka connect client

Steps to run kafka connect on windows7?

IBM MQ and Kafka connector issue (host name supplied is not not valid)

QUESTION

userid is getting changed at the MQ side from kafka connect client

Asked 2019-Sep-08 at 18:32

I'm using https://github.com/ibm-messaging/kafka-connect-mq-source project to build the connector to pull the data from IBM MQ to Kafka topic. I'm seeing an issue where the supplied used-id is getting modified to the system user and authentication failed with the following error :

The MQ Admin reported that I'm supplying different user id. I've configured to send MQUser as userid(I'm executing this connector on linux server). But when it reaches MQ, the userid is modified to wljay (which is the user I logged in on server)

...

ANSWER

Answered 2019-Sep-07 at 21:16

The following line means that the user and password you provide will be sent in the MQCSP structure.

Source https://stackoverflow.com/questions/57828168

QUESTION

Steps to run kafka connect on windows7?

Asked 2018-Sep-02 at 07:02

I am able to setup and run kafka on windows7 as mentioed, can you please help me with the steps to run the mq-connector or other connector jar on windows. thanks in advance

...

ANSWER

Answered 2018-Sep-02 at 07:02

I am answering to my own question, Here are the instructions to Setting Up and Running Apache Kafka on Windows7 OS and steps to run the mq-connector jar or any other connector on windows.

Source https://stackoverflow.com/questions/48314271

QUESTION

IBM MQ and Kafka connector issue (host name supplied is not not valid)

Asked 2018-Jul-10 at 11:42

I have been trying to connect IBM MQ to Kafka on my Ubuntu. I want to get the messages from MQ to Kafka. I am trying to use a connector Link. I followed all the steps, but I keep on getting the following errors:

host name supplied is not valid.

and

JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2538' ('MQRC_HOST_NOT_AVAILABLE').

I tried everything, but nothing seems to work. If anyone has faced the same issue, please let me know.

My configuration includes the following line:-

...

ANSWER

Answered 2018-Apr-27 at 20:29

host name supplied is not valid.

JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2538' ('MQRC_HOST_NOT_AVAILABLE').

The error message is pretty clear, the hostname/connection name is invalid.

That link you gave says that the "mq-source.properties" file has the connection information specified as:

Source https://stackoverflow.com/questions/50044177

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install kafka-connect-mq-source

To enable use of TLS, set the configuration mq.ssl.cipher.suite to the name of the cipher suite which matches the CipherSpec in the SSLCIPH attribute of the MQ server-connection channel. Use the table of supported cipher suites for MQ 9.1 here as a reference. Note that the names of the CipherSpecs as used in the MQ configuration are not necessarily the same as the cipher suite names that the connector uses. The connector uses the JMS interface so it follows the Java conventions. You will need to put the public part of the queue manager's certificate in the JSSE truststore used by the Kafka Connect worker that you're using to run the connector. If you need to specify extra arguments to the worker's JVM, you can use the EXTRA_ARGS environment variable.
You will need to put the public part of the client's certificate in the queue manager's key repository. You will also need to configure the worker's JVM with the location and password for the keystore containing the client's certificate. Alternatively, you can configure a separate keystore and truststore for the connector.

Support

By default, the connector does not use keys for the Kafka messages it publishes. It can be configured to use the JMS message headers to set the key of the Kafka records. You could use this, for example, to use the MQMD correlation identifier as the partitioning key when the messages are published to Kafka. There are four valid values for the mq.record.builder.key.header that controls this behavior. In MQ, the message ID and correlation ID are both 24-byte arrays. As strings, the connector represents them using a sequence of 48 hexadecimal characters.

Find more information at: