go-source | A golang source library

The character you're referring to is typically common with Avro serialized data that's been decoded into a string.

Check the key/value converter settings in the Connect worker since you've not defined it in the Connector.

If you want to parse to JSON, use the JSONConverter, otherwise Avro would work as well if you want to skip data class definitions and generate that from the Avro schema

Local environment, firewall not allowing golang to dial tcp..
It still allowed the url to be resolved to an ip though (DNS)

You've set "key.ignore": "true" which per the docs means that the connector will use the message's topic+partition+offset for the Elasticsearch document ID. Since each update and delete message in Kafka is a new message, you'll be getting a new Elasticsearch document each time.

Set "key.ignore": "true" in your sink connector and make sure that your Kafka message key uniquely identifies the document that you want to update/edit in Elasticsearch.

To handle this specifically with a MongoDB Source, you need to pull the source ID out of the STRUCT in the key by adding this to your source connector:

stage is only supported on replica sets

You need to make your Mongo database a replica set in order to read the oplog

https://dba.stackexchange.com/questions/243780/converting-mongodb-instance-from-standalone-to-replica-set-and-backing-up

Mongo-source doesn't support tasks.max > 1. Even if you set it greater than 1 only one task will be pulling data from mongo to Kafka.

How many task is created depends on particular connector. Function List> Connector::taskConfigs(int maxTasks), (that should be overridden during the implementation of your connector) return the list, which size determine number of Tasks. If you check mongo-kafka source connector you will see, that it is singletonList.

https://github.com/mongodb/mongo-kafka/blob/master/src/main/java/com/mongodb/kafka/connect/MongoSourceConnector.java#L47

From the godep documentation about migration:

dep assumes that all generated code exists, and has been committed to the source.

Therefore, it seems to be impossible to do what I want. The solution is to create a repository which contains the generated sources, and make sure these sources are automatically generated and kept in-sync with the actual source data (in my case the raw *.proto files).

Since I cannot put the generated sources into the same repository as the source data, it is neccessary to completely synchronise these two repositories (same branches, same tags), so that the versions used by go dep are somehow useful when comparing with the actual repository, that only contains declarations.