Follower Fetching
Learn about follower fetching and how to configure a Redpanda consumer to fetch records from the closest replica.
About follower fetching
Follower fetching enables a consumer to fetch records from the closest replica of a topic partition, regardless of whether it’s a leader or a follower.
For a Redpanda cluster deployed across different data centers and availability zones (AZs), restricting a consumer to fetch only from the leader of a partition can incur greater costs and have higher latency than fetching from a follower that is geographically closer to the consumer.
With follower fetching (proposed in KIP-392), the fetch protocol is extended to support a consumer fetching from any replica. This includes Remote Read Replicas.
The first fetch from a consumer is processed by a Redpanda leader broker. The leader checks for a replica (itself or a follower) that has a rack ID that matches the consumer’s rack ID. If a replica with a matching rack ID is found, the fetch request returns records from that replica. Otherwise, the fetch is handled by the leader.
Configure follower fetching
Redpanda decides which replica a consumer fetches from. If the consumer configures its client.rack
property, Redpanda by default selects a replica from the same rack as the consumer, if available.
To enable follower fetching in Redpanda, configure properties for the consumer and the Redpanda cluster and broker:
-
For a Redpanda cluster, set the
enable_rack_awareness
property totrue
. -
For each Redpanda broker, set the
rack
property to a rack ID. -
For each consumer, set the
client.rack
property to a rack ID.