kafka-wurstmeister-image-01

Mark
docker-compose.yml
version: '3'

services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
  • services – configures the services that will be executed in separate containers. Here, we have configured two services zookeeper and kafka
  • image – defines the base image or snapshot to build the container. Here, we have used wurstmeister/zookeeper and wurstmeister/kafka images to build their respective containers.
  • container_name – the name of containers.
  • ports – specifies the exposing point of containers.
  • environment – defines the environment variables for the container. Here, we have set KAFKA_ADVERTISED_HOST_NAME and KAFKA_ZOOKEEPER_CONNECT for the kafka container.

Start Containers

docker-compose up -d
Output
...
Status: Downloaded newer image for wurstmeister/kafka:latest
Creating zookeeper ... done
Creating kafka     ... done

Start the kafka container interactive shell

docker exec -it kafka /bin/sh
cd /opt
ls

… in our case:

kafka  kafka_2.13-2.8.1  overrides

Use your version

# cd kafka_2.13-2.8.1/bin
# ls

connect-distributed.sh
connect-mirror-maker.sh
connect-standalone.sh
kafka-acls.sh
kafka-broker-api-versions.sh
kafka-cluster.sh
kafka-configs.sh
kafka-console-consumer.sh
kafka-console-producer.sh
kafka-consumer-groups.sh
kafka-consumer-perf-test.sh
kafka-delegation-tokens.sh
kafka-delete-records.sh
kafka-dump-log.sh
kafka-features.sh
kafka-leader-election.sh
kafka-log-dirs.sh
kafka-metadata-shell.sh
kafka-mirror-maker.sh
kafka-preferred-replica-election.sh
kafka-producer-perf-test.sh
kafka-reassign-partitions.sh
kafka-replica-verification.sh
kafka-run-class.sh
kafka-server-start.sh
kafka-server-stop.sh
kafka-storage.sh
kafka-streams-application-reset.sh
kafka-topics.sh
kafka-verifiable-consumer.sh
kafka-verifiable-producer.sh
trogdor.sh
zookeeper-security-migration.sh
zookeeper-server-start.sh
zookeeper-server-stop.sh
zookeeper-shell.sh
#

Create New Topic

kafka-topics.sh

kafka-topics.sh --create --zookeeper zookeeper:2181 --replication-factor 1 --partitions 1 --topic Demo-topic

Created topic Demo-topic.

Verification

kafka-topics.sh

kafka-topics.sh --list --zookeeper zookeeper:2181

Demo-topic

Run Producer to Generate Messages

kafka-console-producer.sh

kafka-console-producer.sh --topic Demo-topic --bootstrap-server localhost:9092
>Hello
>jreact.com
>Using Docker Compose
>^C#

Run Consumer

kafka-console-consumer.sh

kafka-console-consumer.sh --topic Demo-topic --from-beginning --bootstrap-server localhost:9092

Hello
jreact.com
Using Docker Compose
^CProcessed a total of 3 messages

–

Mark