Skip to main content
Version: 22.3

Enabling Rack Awareness in Manual Deployments

Rack awareness allows you to distribute replicas of the same partition across different racks to minimize data loss in the event of a rack failure. A rack is a failure zone that has one or more Redpanda brokers assigned to it.

When you create a topic, you specify the number of partitions for the topic and the number of partition replicas. By default, Redpanda determines where to place the replicas on the cluster such that each replica is on a different broker, if possible.

By defining different racks for a Redpanda cluster, you can specify a preference for the way partition replicas are assigned to brokers. When Redpanda places partition replicas, it takes into account whether a replica has already been placed on a broker in a particular rack. If so, Redpanda chooses a broker in a different rack. This way, partition replicas are distributed across different failure zones, which provides a measure of fault tolerance in the event that a broker or an entire rack becomes unavailable.

When rack awareness is enabled, Redpanda places replicas according to these criteria:

  • Number of racks vs. replicas - If the cluster has more racks than the number of replicas, each replica is placed on a broker in a unique rack. If the cluster has fewer racks than the number of replicas, some replicas are placed on brokers in the same rack.
  • Number of available CPU cores - Brokers with more available CPU cores are chosen over brokers with fewer available CPU cores.
  • Broker utilization - Brokers with fewer partitions are chosen over brokers with more partitions.

Setting up rack awareness

The instructions in this section are based on a cluster with five brokers and three failure zones. The failure zones, or racks, are identified as A, B, and C. The brokers are assigned to the three racks as shown:

BrokerRack
1A
2A
3B
4C
5C

To set up rack awareness for this cluster:

  1. Open a terminal window and log in to broker 1 on the cluster.
  2. Run rpk cluster config edit.
  3. In the text editor window that opens, set enable_rack_awareness to true.
  4. Save your change and quit the editor.
  5. Go to the /etc/redpanda directory.
  6. Open the redpanda.yaml file for editing.
  7. In the redpanda section, add the rack property and set the value to A as shown:
    rack: "A"
  8. Log in to the remaining brokers and edit their redpanda.yaml files. For broker 2, set rack to A. For broker 3, set rack to B. For brokers 4 and 5, set rack to C.
  9. Restart each broker in the cluster for the rack assignments to take effect.
tip

For high availability, Redpanda recommends adding one or more brokers from each zone to the seed_servers configuration.

Next steps

Use rack awareness with Continuous Data Balancing to continually maintain the configured replication level, even after a rack failure. For a given partition, Redpanda tries to move excess replicas from racks that have more than one replica to racks that have no replicas.

What do you like about this doc?




Optional: Share your email address if we can contact you about your feedback.

Let us know what we do well: