Scale Redpanda in Kubernetes

You can scale a cluster both vertically, by increasing or decreasing the resources available to existing brokers, and horizontally, by increasing or decreasing the number of brokers in the cluster.

Vertical scaling

Vertical scaling involves increasing the amount of resources available to Redpanda brokers (scaling up) or decreasing the amount of resources (scaling down). Resources include the amount of hardware available to Redpanda brokers, such as CPU cores, memory, and storage.

To scale a Redpanda cluster vertically, see Manage Pod Resources in Kubernetes.

You cannot decrease the number of CPU cores in a running cluster.

If your existing worker nodes have either too many resources or not enough resources, you may need to move Redpanda brokers to new worker nodes that meet your resource requirements. This process involves:

  • Making sure the new worker nodes are available.

  • Deleting each worker node one by one.

  • Deleting the Pod’s PersistentVolumeClaim (PVC).

  • Ensuring that the PersistentVolume’s (PV) reclaim policy is set to Retain to make sure that you can roll back to the original worker node without losing data.

For emergency scenarios in which a node unexpectedly fails or is decommissioned without warning, the Nodewatcher controller can help protect your Redpanda data. For details, see Install the Nodewatcher Controller.

Horizontal scaling

Horizontal scaling involves modifying the number of brokers in your cluster, either by adding new ones (scaling out) or removing existing ones (scaling in). In situations where the workload is variable, horizontal scaling allows for flexibility. You can scale out when demand is high and scale in when demand is low, optimizing resource usage and cost.

Redpanda does not support Kubernetes autoscalers. Autoscalers rely on CPU and memory metrics for scaling decisions, which do not fully capture the complexities involved in scaling Redpanda clusters. Always manually scale your Redpanda clusters as described in this topic.

While you should not rely on Kubernetes autoscalers to scale your Redpanda brokers, you can prevent infrastructure-level autoscalers like Karpenter from terminating nodes that run Redpanda Pods. For example, you can set the statefulset.podTemplate.annotations field in the Redpanda Helm values, or the statefulset.podTemplate.annotations field in the Redpanda custom resource to include:

karpenter.sh/do-not-disrupt: "true"

This annotation tells Karpenter not to disrupt the node on which the annotated Pod is running. This can help protect Redpanda brokers from unexpected shutdowns in environments that use Karpenter to manage infrastructure nodes.

Scale out

Scaling out is the process of adding more brokers to your Redpanda cluster. You may want to add more brokers for increased throughput, high availability, and fault tolerance. Adding more brokers allows for better distribution of data across the cluster. This can be particularly important when dealing with large data sets.

To add Redpanda brokers to your cluster:

  1. Ensure that you have one additional worker node for each Redpanda broker that you want to add. Each Redpanda broker requires its own dedicated worker node so that it has access to all resources. For more details, see Kubernetes Cluster Requirements and Recommendations.

  2. If you use local PersistentVolumes (PV), ensure that your additional worker nodes have local disks available that meet the requirements of the configured StorageClass. See Store the Redpanda Data Directory in PersistentVolumes.

  3. If you have external access enabled, make sure that your new node has the necessary node ports open to external clients. See Networking and Connectivity in Kubernetes.

  4. Verify that your cluster is in a healthy state:

    kubectl exec redpanda-0 --namespace <namespace> -- rpk cluster health

  5. Increase the number of replicas in the Helm values:

    • Helm + Operator

    • Helm

    redpanda-cluster.yaml
    apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  chartRef: {}
  clusterSpec:
    statefulset:
      replicas: <number-of-replicas>
    kubectl apply -f redpanda-cluster.yaml --namespace <namespace>

    • --values

    • --set

    replicas.yaml
    statefulset:
  replicas: <number-of-replicas>
    helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
  --values replicas.yaml --reuse-values
    helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
  --set statefulset.replicas=<number-of-replicas>

  6. Wait until the StatefulSet is rolled out:

    kubectl --namespace <namespace> rollout status statefulset redpanda --watch

  7. Verify that your cluster is in a healthy state:

    kubectl exec redpanda-0 --namespace <namespace> -- rpk cluster health

Scale in

Scaling in is the process of removing brokers from your Redpanda cluster. You may want to remove brokers for cost reduction and resource optimization.

To scale in a Redpanda cluster, follow the instructions for decommissioning brokers in Kubernetes to safely remove brokers from the Redpanda cluster.