Kafka Consumer Groups CLI Demo KafkaPython

YOUR LINK HERE:

http://youtube.com/watch?v=yKzdum_vX7k

This video demonstrates the Kafka Consumer Groups concept using CLI • Prerequisite: • -------------------------- • Consumer Consumer Group in Kafka • • Consumer Consumer Group in Kafka • CLI Codes used • (this will only work if you installed kafka following the video: • Install and run Apache Kafka integr... ) • Start Zookeeper: • --------------------------------------- • F:/kafka_2.12-3.2.0/bin/windows/zookeeper-server-start.bat F:/kafka_2.12-3.2.0/config/zookeeper.properties • Start Kafka-server: • ----------------------------------------- • F:/kafka_2.12-3.2.0/bin/windows/kafka-server-start.bat F:/kafka_2.12-3.2.0/config/server.properties • Create topic: • ------------------------------------ • F:/kafka_2.12-3.2.0/bin/windows/kafka-topics.bat --create --topic hello_world1 --bootstrap-server localhost:9092 --replication-factor 1 --partitions 3 • • Without Consumer Group: • -------------------------- • F:/kafka_2.12-3.2.0/bin/windows/kafka-console-consumer.bat --topic hello_world1 --from-beginning --bootstrap-server localhost:9092 • • F:/kafka_2.12-3.2.0/bin/windows/kafka-console-consumer.bat --topic hello_world1 --from-beginning --bootstrap-server localhost:9092 • • Start Consumer with Group: • -------------------------------------------------------- • F:/kafka_2.12-3.2.0/bin/windows/kafka-console-consumer.bat --topic hello_world1 --from-beginning --bootstrap-server localhost:9092 --group my-first-consumer-group • Producer Code: • -------------------------------- • from time import sleep • from json import dumps • from kafka import KafkaProducer • topic_name='hello_world1' • producer = KafkaProducer(bootstrap_servers=['localhost:9092'],value_serializer=lambda x: dumps(x).encode('utf-8')) • while True: • message=input( Enter the message you want to send : ) • partition_no=int(input( In which partition you want to send ? )) • producer.send(topic_name, value=message,partition=partition_no) • producer.close() • Check this playlist for more Data Engineering related videos: • • Demystifying Data Engineering with Cl... • Snowflake Complete Course from scratch with End-to-End Project with in-depth explanation-- • https://doc.clickup.com/37466271/d/h/... • 🙏🙏🙏🙏🙏🙏🙏🙏 • YOU JUST NEED TO DO • 3 THINGS to support my channel • LIKE • SHARE • • SUBSCRIBE • TO MY YOUTUBE CHANNEL

#############################

New on site