Learn about follower fetching and how to configure a Redpanda consumer to fetch records from the closest replica.
Follower fetching enables a consumer to fetch records from the closest replica of a topic partition, regardless of whether it’s a leader or a follower.
For a Redpanda cluster deployed across different datacenters and availability zones (AZs), restricting a consumer to fetch only from the leader of a partition can incur greater costs and have higher latency than fetching from a follower that is geographically closer to the consumer.
With follower fetching (proposed in KIP-392), the fetch protocol is extended to support a consumer fetching from any replica.
The first fetch from a consumer is processed by a Redpanda leader node. The leader checks for a replica (itself or a follower) that has a rack ID that matches the consumer’s rack ID. If a replica with a matching rack ID is found, the fetch request returns records from that replica. Otherwise, the fetch is handled by the leader.
Redpanda decides which replica a consumer fetches from. If the consumer configures its
client.rack property, Redpanda by default selects a replica from the same rack as the consumer, if available.
To enable follower fetching in Redpanda, configure properties for the consumer and the Redpanda cluster and node:
For each consumer, set the
client.rack property to a rack ID. Rack awareness is pre-enabled for cloud-based clusters in multi-AZ environments.