Today , 28th March ,2020 - What I have learnt today
handwritten notes available here, if you interest please check
On my lock down days here at my home. Today I have started to learn more about Apache Kafka.
Here is my notes to recollect all I have memorised and understood.
Kafka is 3rd generation messaging system or often called streaming platform (What is Messaging System ? - Passing the messages between computers or distributed applications in asynchronously, securely and scale able fashion.
It will allows you to store and process the streaming of data/message as they occur.
Kafka will store the messages in Topic those were received from or published by Producers.
Producers can be an any application to send the messages to Kafka Cluster.
Consumers are nothing but a client application which consumes the data / messages from Kafka Cluster.
Streams : Processors , Its an client library for processing the data or messages in Kafka.
Connectors : A Framework to build connectors which is responsible for moving or transporting the data or messages in and out or other systems from Kafka.
Topic : Its a unique for Kafka system, its a logical container to hold or store the messages from the producers.
Partition : Kafka breaks the messages into multiple partition and storing it in host of computers in order to comply the fault-tolerance and high availability.
Support a kafka Topic has huge data, then these data will be split into partition and each messages will be stored in the cell format and cell will share the sequence of id, These id will be called OffSet in Kafkar.
Messages will be identified or retrieved through this Offset Id and combination of Topic Name,Partition Id.