Enable Rack Awareness

Rack awareness allows you to distribute replicas of the same partition across different racks to minimize data loss in the event of a rack failure. A rack is a failure zone that has one or more Redpanda brokers assigned to it.

When you create a topic, you specify the number of partitions for the topic and the number of partition replicas. By default, Redpanda determines where to place the replicas on the cluster such that each replica is on a different broker, if possible.

By defining different racks for a Redpanda cluster, you can specify a preference for the way partition replicas are assigned to brokers. When Redpanda places partition replicas, it takes into account whether a replica has already been placed on a broker in a particular rack. If so, Redpanda chooses a broker in a different rack. This way, partition replicas are distributed across different failure zones, which provides a measure of fault tolerance in the event that a broker or an entire rack becomes unavailable.

When rack awareness is enabled, Redpanda places replicas according to these criteria:

  • Number of racks vs. replicas - If the cluster has more racks than the number of replicas, each replica is placed on a broker in a unique rack. If the cluster has fewer racks than the number of replicas, some replicas are placed on brokers in the same rack.

  • Number of available CPU cores - Brokers with more available CPU cores are chosen over brokers with fewer available CPU cores.

  • Broker utilization - Brokers with fewer partitions are chosen over brokers with more partitions.

Configure rack awareness

The instructions in this section are based on a cluster with six brokers and three failure zones. The failure zones, or racks, are identified as A, B, and C, with brokers assigned to them:

Broker Rack

1

A

2

A

3

B

4

B

5

C

6

C

To set up rack awareness for this cluster:

  1. Open a terminal window and log in to broker 1 on the cluster.

  2. Run rpk cluster config edit.

  3. In the text editor window that opens, set enable_rack_awareness to true.

  4. Save your change and quit the editor.

  5. Go to the /etc/redpanda directory.

  6. Open the redpanda.yaml file for editing.

  7. In the redpanda section, add the rack property and set the value to A as shown:

    rack: "A"

  8. Log in to the remaining brokers and edit their redpanda.yaml files. For broker 2, set rack to A. For brokers 3 and 4, set rack to B. For brokers 5 and 6, set rack to C.

  9. Restart each broker in the cluster for the rack assignments to take effect.

For high availability, Redpanda recommends adding one or more brokers from each zone to the seed_servers configuration.

Redpanda supports uneven rack assignments. For example, you may have four or five brokers to assign across three racks.

An uneven assignment might cause undesirable effects:

  • The local storage in some racks will be more full than in others, which is generally less cost-efficient.

  • The compute capacity is also uneven across racks, which could result in uneven latency in some cases.

An equal number of brokers assigned to each rack helps to avoid these scenarios, and is therefore considered optimal.

Next steps

Use rack awareness with Continuous Data Balancing to continually maintain the configured replication level, even after a rack failure. For a given partition, Redpanda tries to move excess replicas from racks that have more than one replica to racks that have no replicas.