remote kafka


The official demo, kafka.apache.org/quickstart, shows how to run kafka with everything on localhost.
Here are the steps to run kafka server remotely.
tech.riowing.net is the server, which runs kafka zookeeper and brokers on ubuntu in aws
WSL, a windows desktop, runs kafka producer and consumer on WSL ubuntu on local windows

On both AWS and WSL:
download kafka tar ball and extract:
http://apache.claz.org/kafka/2.3.0/kafka_2.12-2.3.0.tgz 57M
extract to: /mnt/ebs/rio/opensrc/kafka/kafka_2.12-2.3.0
install java. I got OpenJDK 64-Bit Server VM build 11

On AWS, server side:
export KAFKA_OPTS=”-Djava.net.preferIPv4Stack=True”
otherwise it’s IPV6
edit server.properties so that:
advertised.listeners=PLAINTEXT://tech.riowing.net:9092
otherwise client report error: Connection to node 0 (localhost/127.0.0.1:9092)
start zookeepter:
bin/zookeeper-server-start.sh config/zookeeper.properties
start server:
bin/kafka-server-start.sh config/server.properties

On WSL, client side:
Send message:
bin/kafka-console-producer.sh –broker-list tech.riowing.net:9092 –topic test
type some text, ctrlD to exit
Check message received:
bin/kafka-console-consumer.sh –bootstrap-server tech.riowing.net:9092 –topic test –from-beginning
ctrlC to exit

Previous Post
Next Post
Comments are closed.
%d bloggers like this: