For many years, Apache Kafka relied on Apache ZooKeeper to manage metadata, cluster configurations and maintain a distributed state across clusters. ZooKeeper was crucial in handling essential aspects such as controller election and tracking broker states. However, Kafka recently shifted away from ZooKeeper, opting for a new internal KRaft (Kafka Raft) consensus protocol, marking a significant evolution in Kafka’s architecture. Let’s explore the motivations behind this shift, the benefits it brings, and how KRaft fundamentally changes Kafka.
Why Kafka ditched Zookeeper
