docker-kafka | Docker image containing a standard Kafka distribution | Pub Sub library

i think zookeeper cannot find kafka and kafka cannot find zookeeper becouse localHost name conflict if use Ubuntu OS used this way? if Ubuntu i think you need change setup hosts from /etc/hosts

look this link http://manpages.ubuntu.com/manpages/bionic/man5/hosts.5.html

and this is soultion

Your shell scripts work while you're in the container, but that's not going to help you test code running outside of Docker

You seem to have the internal and external listeners flipped

From the host, you want to connect on the internal listener, so you need to expose port 9093 and connect on localhost:9093, not use the Kafka container service name

To establish connection with Bitnami Docker Kafka running on `localhost, add the following environment variables:

• KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
• KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092

docker-compose.yml kafka service:

Kafka doesn't run HTTP server. Port 9092 over wget doesn't do anything

You need to describe the topics using the kafka-topics command, which can be found in the documentation, which will connect to Zookeeper over the port 2181.

Something like

KAFKA_ADVERTISED_HOST_NAME: localhost

is what your client will receive as connection string to Kafka. It means that container with your app will try to reach Kafka at localhost where localhost is local network stack for container with your app.

To solve that problem you should use kafka host as your KAFKA_ADVERTISED_HOST_NAME and bootstrap servers - that will allow your container to connect to kafka inside dockers but will break configuration when you try to connect to Kafka by running on your PC java -jar...

To fix that problem make entry in your hosts file on your PC with mapping kafka -> localhost.

Maybe I'm just clashing into some opinionated docker image and should look for a different image...

The image is fine. You might want to read this explanation of the listeners.

tl;dr - you don't want to (and shouldn't?) use the same listener "protocol" in different networks.

Use the advertised.listeners, no need to edit the listeners

After a long search, I found the problem. When I run the ches/kafka which is the docker kafka container that I use, I did not specified a port number.

I have figured out that VM did not have enough memory to allocate for kafka server. I got it from the last 3 bottom lines of the second picture above. And the solution is quite easy, I just need to assign bigger memory to VM in vagrantfile, previously it was 1024. And now:

A global config log.retention.{hours|minutes|ms} controls the default behavior for all topics. The topic-level configretention.ms` is for individual topics.

In your case, the log did not get deleted before you set the topic-level config since the default parameter value is 7 days.