Follower Fetching

Learn about follower fetching and how to configure a Redpanda consumer to fetch records from the closest replica.

About follower fetching

Follower fetching enables a consumer to fetch records from the closest replica of a topic partition, regardless of whether it’s a leader or a follower.

For a Redpanda cluster deployed across different datacenters and availability zones, restricting a consumer to fetch only from the leader of a partition can incur more costs and have higher latency than fetching from a follower that is geographically closer to the consumer.

With follower fetching (proposed in KIP-392), the fetch protocol is extended to support a consumer fetching from any replica.

The first fetch from a consumer is processed by a Redpanda leader node. The leader checks for a replica (itself or a follower) that has a rack ID that matches the consumer’s rack ID. If a replica with a matching rack ID is found, the fetch request returns records from that replica. Otherwise, the fetch is handled by the leader.

Configure follower fetching

Redpanda decides which replica a consumer fetches from. If the consumer configures its client.rack property, Redpanda by default selects a replica from the same rack as the consumer, if available.

To enable follower fetching in Redpanda, configure properties for the consumer and the Redpanda cluster and node:

  • For a Redpanda cluster, set the enable_rack_awareness property to true.

  • For each Redpanda node, set the rack property to a rack ID.

  • For each consumer, set the client.rack property to a rack ID.