Leader Pinning

Produce requests that write data to Redpanda topics go through the topic partition leader, which syncs messages across its follower replicas. For a Redpanda cluster deployed across multiple availability zones (AZs), leader pinning ensures that a topic’s partition leaders are geographically closer to clients, which helps decrease networking costs and guarantees lower latency.

If consumers are located in the same preferred region or AZ for leader pinning, and you have not set up follower fetching, leader pinning can also help reduce networking costs on consume requests.

Configure leader pinning

Configure leader pinning if you have Redpanda deployed in a multi-AZ or multi-region cluster and your ingress is concentrated in a particular AZ or region.

Use the topic configuration property redpanda.leaders.preference to configure leader pinning for individual topics. The property accepts the following string values:

  • none: Opt out the topic from leader pinning.

  • racks:<rack1>[,<rack2>,…​]: Specify the preferred location (rack) of all topic partition leaders. The list can contain one or more racks, and you can list the racks in any order. Spaces in the list are ignored, for example: racks:rack1,rack2 and racks: rack1, rack2 are equivalent. You cannot specify empty racks, for example: racks: rack1,,rack2. If you specify multiple racks, Redpanda tries to distribute the partition leader locations equally across brokers in these racks.

    To find the rack identifier, run rpk cluster info.

If there is more than one broker in the preferred AZ (or AZs), leader pinning distributes partition leaders uniformly across brokers in the AZ.

Leader pinning failover across availability zones

If there are three AZs: A, B, and C, and A becomes unavailable, the failover behavior is as follows:

  • A topic with "A" as the preferred leader AZ will have its partition leaders uniformly distributed across B and C.

  • A topic with "A,B" as the preferred leader AZs will have its partition leaders in B.

  • A topic with “B” as the preferred leader AZ will have its partition leaders in B as well.

Suggested reading