Statement : While Designing the high scale systems , most of the times we need to queue the requests to serve it in async fashion and for that we need some scalable messaging system which can handle this much of load in Distributed Computing. On that note, this article is going to introduce you with one of the de facto industry standard messaging system that is Apache Kafka and a framework written on the top of Kafka that is Kafka Connect.
Fundamentals Covered as a part of this article :
– Kafka Topics
– Records
– Partitions
– Replications
– Offset Management
– Zookeeper
– Producer APIs
– Consumer APIs
– Streams APIs
– Producer Consumer Model
– Source Connectors
– Sink Connectors
Visual Representation of Apache Kafka and Kafka Connect
References : https://kafka.apache.org/ https://docs.confluent.io/3.0.0/connect/
Finally, I have covered everything discussed above as a part of my youtube video. In this video I am going to cover the basics of Apache Kafka and Kafka Connect in 10 minutes. In addition to it, You will visualize the beauty of various connectors used in industry by leveraging Apache Kafka and Kafka Connect. Hopefully by watching this video, you can easily go into the depth of Kafka and implement it in your products/ projects/ modules as a part of messaging system. Feel free to like the video and share your valuable feedback.
Last but not least, please don’t forget to subscribe my youtube channel. Stay updated and connected. Cheers 🙂
D Technologist Geek 