bookmark_borderBrokers in Kafka

Kafka Cluster

The central component of Kafka architecture is the broker.

Brokers are the servers that make up a Kafka cluster (one or more brokers).

Producers and consumers communicate with brokers in order to publish and consume messages.

Zookeeper

Kafka depends on an underlying technology called Zookeeper.

Zookeeper is a generalized cluster management tool. It manages the cluster and provides a consistent, distributed place to store cluster configuration.

Zookeeper coordinates communication throughout the cluster adds and removes brokers, and monitors the status of nodes in the cluster. It is often installed alongside Kafka but can be maintained on a completely separate set of servers.

Networking

Kafka uses a simple TCP protocol to handle messaging communication.

The Controller

In a Kafka cluster, one broker is dynamically designated as the controller. The controller coordinates the process of assigning partitions and data replicas to nodes in the cluster.

Every cluster has exactly one controller. If the controller goes down, another node will automatically become the controller.

ANOTE.DEV