# Deploy Redpanda for Production in Kubernetes

> For the complete documentation index, see [llms.txt](https://docs.redpanda.com/llms.txt). Component-specific: [streaming-full.txt](https://docs.redpanda.com/streaming-full.txt)

---
title: Deploy Redpanda for Production in Kubernetes
latest-redpanda-tag: v25.3.11
latest-console-tag: v3.7.3
latest-operator-version: v26.1.4
# EOL = End-of-Life (support lifecycle status)
page-is-nearing-eol: "false"
page-is-past-eol: "false"
page-eol-date: November 19, 2026
latest-connect-version: 4.93.0
docname: redpanda/kubernetes/k-production-deployment
page-component-name: streaming
page-version: "25.3"
page-component-version: "25.3"
page-component-title: Streaming
page-relative-src-path: redpanda/kubernetes/k-production-deployment.adoc
page-edit-url: https://github.com/redpanda-data/docs/edit/v/25.3/modules/deploy/pages/redpanda/kubernetes/k-production-deployment.adoc
description: Deploy a Redpanda cluster in Kubernetes.
page-git-created-date: "2025-08-15"
page-git-modified-date: "2026-02-06"
support-status: supported
---

<!-- Source: https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-production-deployment.md -->

This topic describes how to configure and deploy one or more Redpanda clusters and Redpanda Console in Kubernetes.

## [](#prerequisites)Prerequisites

Make sure that your Kubernetes cluster meets the [requirements](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-requirements/).

You must already have a ConfigMap that stores your `io-config.yaml` file. See [Generate optimal I/O configuration settings](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-tune-workers/#io).

## [](#deploy-a-redpanda-cluster)Deploy a Redpanda cluster

To deploy Redpanda and Redpanda Console, you can use the following tools:

-   **Redpanda Operator**: The Redpanda Operator extends Kubernetes with custom resource definitions (CRDs), allowing you to define Redpanda clusters as native Kubernetes resources. The resource that the Redpanda Operator uses to represent a Redpanda cluster is the Redpanda resource. The Redpanda Operator can be deployed in either cluster scope (managing resources across all namespaces) or namespace scope (managing resources within a single namespace).

-   **Helm**: [Helm](https://helm.sh/docs) is a package manager for Kubernetes, which simplifies the process of defining, installing, and upgrading Kubernetes applications. Helm uses charts, a collection of files that describe a related set of Kubernetes resources, to deploy applications in a Kubernetes cluster.


> 💡 **TIP**
>
> For more details about the differences between these two methods, see [Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-deployment-overview/).

### Operator

The Redpanda Operator can be deployed in two different scopes:

-   **Cluster scope** (recommended): The Redpanda Operator manages Redpanda resources across all namespaces in your Kubernetes cluster. This provides centralized management and is ideal for production environments.

-   **Namespace scope**: The Redpanda Operator manages Redpanda resources only within a specific namespace. This provides better isolation and is suitable when you need strict namespace boundaries.


> ⚠️ **WARNING**
>
> Do not run multiple Redpanda Operators in different scopes (cluster and namespace scope) in the same cluster as this can cause resource conflicts.

1.  Make sure that you have permission to install custom resource definitions (CRDs):

    ```bash
    kubectl auth can-i create CustomResourceDefinition --all-namespaces
    ```

    You should see `yes` in the output.

    You need these cluster-level permissions to install [cert-manager](https://cert-manager.io/docs/) and Redpanda Operator CRDs in the next steps.

2.  Install [cert-manager](https://cert-manager.io/docs/installation/helm/):

    ```bash
    helm repo add jetstack https://charts.jetstack.io
    helm repo update
    helm install cert-manager jetstack/cert-manager \
      --set crds.enabled=true \
      --namespace cert-manager  \
      --create-namespace
    ```

    The Redpanda Helm chart enables TLS by default and uses cert-manager to manage TLS certificates.

3.  Deploy the Redpanda Operator in your chosen scope:

    1.  To deploy in cluster scope, use:

        ```bash
        helm repo add redpanda https://charts.redpanda.com
        helm repo update
        helm upgrade --install redpanda-controller redpanda/operator \
          --namespace <namespace> \
          --create-namespace \
          --version v26.1.4 \ (1)
          --set crds.enabled=true (2)
        ```

        | 1 | This flag specifies the exact version of the Redpanda Operator Helm chart to use for deployment. By setting this value, you pin the chart to a specific version, which prevents automatic updates that might introduce breaking changes or new features that have not been tested in your environment. |
        | --- | --- |
        | 2 | This flag ensures that the CRDs are installed as part of the Redpanda Operator deployment.This command deploys the Redpanda Operator in cluster scope (default in v25.2+), allowing it to manage Redpanda clusters across multiple namespaces. |

    2.  To deploy in namespace scope (managing only resources within its deployment namespace), use:

        ```bash
        helm upgrade --install redpanda-controller redpanda/operator \
          --namespace <namespace> \
          --create-namespace \
          --version v26.1.4 \
          --set crds.enabled=true \
          --set 'additionalCmdFlags=["--namespace=<cluster-namespace>"]' (1)
        ```

        | 1 | This flag restricts the Redpanda Operator to manage resources only within the specified namespace. |
        | --- | --- |


4.  Ensure that the Deployment is successfully rolled out:

    ```bash
    kubectl --namespace <namespace> rollout status --watch deployment/redpanda-controller-operator
    ```

    deployment "redpanda-controller-operator" successfully rolled out

5.  Install a [Redpanda custom resource](https://docs.redpanda.com/streaming/25.3/reference/k-crd/) to deploy a Redpanda cluster and Redpanda Console.

    `redpanda-cluster.yaml`

    ```yaml
    apiVersion: cluster.redpanda.com/v1alpha2
    kind: Redpanda
    metadata:
      name: redpanda
      namespace: <cluster-namespace>
    spec:
      clusterSpec:
        #enterprise:
          #licenseSecretRef:
            #name: <secret-name>
            #key: <secret-key>
        image:
          tag: v25.3.11
        statefulset:
          extraVolumes: |-
            - name: redpanda-io-config
              configMap:
                name: redpanda-io-config
          extraVolumeMounts: |-
            - name: redpanda-io-config
              mountPath: /etc/redpanda-io-config
          additionalRedpandaCmdFlags:
            - "--io-properties-file=/etc/redpanda-io-config/io-config.yaml"
    ```

    -   `metadata.name`: Name to assign the Redpanda cluster.

    -   `metadata.namespace`: For cluster-scoped deployment, specify any namespace. For namespace-scoped deployment, must be the same namespace where the Redpanda Operator is deployed.

    -   [`spec.clusterSpec`](https://docs.redpanda.com/streaming/25.3/reference/k-crd/#k8s-api-github-com-redpanda-data-redpanda-operator-api-redpanda-v1alpha2-redpandaclusterspec): This is where you can override default values in the Redpanda Helm chart. Here, you mount the [I/O configuration file](#prerequisites) to the Pods that run Redpanda. For other configuration details, see [Production considerations](#config).

    -   `spec.clusterSpec.enterprise`: If you want to use enterprise features in Redpanda, uncomment this section and add the details of a Secret that stores your Enterprise Edition license key. For details, see [Redpanda Licensing](https://docs.redpanda.com/streaming/25.3/get-started/licensing/).

    -   `spec.clusterSpec.image.tag`: Deploys the latest version of Redpanda.

    -   `spec.clusterSpec.statefulset`: Here, you mount the [I/O configuration file](#prerequisites) to the Pods that run Redpanda. For other configuration details, see [Production considerations](#config).


6.  Apply the Redpanda resource:

    ```bash
    kubectl apply -f redpanda-cluster.yaml
    ```

7.  Wait for the Redpanda Operator to deploy Redpanda using the Helm chart:

    ```bash
    kubectl get redpanda --namespace <cluster-namespace> --watch
    ```

    NAME       READY   STATUS
    redpanda   True    Redpanda reconciliation succeeded

    This step may take a few minutes. You can watch for new Pods to make sure that the deployment is progressing:

    ```bash
    kubectl get pod --namespace <cluster-namespace>
    ```

    If it’s taking too long, see [Troubleshooting](https://docs.redpanda.com/streaming/25.3/troubleshoot/errors-solutions/k-resolve-errors/).

8.  Verify that each Redpanda broker is scheduled on only one Kubernetes node:

    ```bash
    kubectl get pod --namespace <cluster-namespace>  \
      -o=custom-columns=NODE:.spec.nodeName,NAME:.metadata.name -l \
      app.kubernetes.io/component=redpanda-statefulset
    ```

    Expected output:

    example-worker3   redpanda-0
    example-worker2   redpanda-1
    example-worker    redpanda-2

### Helm

1.  Install cert-manager using Helm:

    ```bash
    helm repo add jetstack https://charts.jetstack.io
    helm repo update
    helm install cert-manager jetstack/cert-manager \
      --set crds.enabled=true \
      --namespace cert-manager  \
      --create-namespace
    ```

    The Redpanda Helm chart enables TLS by default and uses cert-manager to manage TLS certificates.

2.  Override the default values to mount your [I/O configuration file](#prerequisites) onto each Pod that runs Redpanda.

    `redpanda-values.yaml`

    ```yaml
    image:
      tag: {latest-redpanda-tag}
    statefulset:
      extraVolumes: |-
        - name: redpanda-io-config
          configMap:
            name: redpanda-io-config
      extraVolumeMounts: |-
        - name: redpanda-io-config
          mountPath: /etc/redpanda-io-config
      additionalRedpandaCmdFlags:
        - "--io-properties-file=/etc/redpanda-io-config/io-config.yaml"
    ```

    Redpanda reads from this file at startup to optimize itself for the given I/O parameters.

    If you want to use enterprise features in Redpanda, add the details of a Secret that stores your Enterprise Edition license key.

    `redpanda-values.yaml`

    ```yaml
    enterprise:
      licenseSecretRef:
        name: <secret-name>
        key: <secret-key>
    ```

    For details, see [Add an Enterprise Edition License to Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/get-started/licensing/add-license-redpanda/kubernetes/).

3.  Install the Redpanda Helm chart to deploy a Redpanda cluster and Redpanda Console.

    ```bash
    helm repo add redpanda https://charts.redpanda.com
    helm repo update
    helm install redpanda redpanda/redpanda \
      --version 26.1.4 \ (1)
      --namespace <namespace> \ (2)
      --create-namespace \
      --values redpanda-values.yaml
    ```

    | 1 | This flag specifies the exact version of the Redpanda Helm chart to use for deployment. By setting this value, you pin the chart to a specific version, which prevents automatic updates that might introduce breaking changes or new features that have not been tested in your environment. |
    | --- | --- |
    | 2 | Each deployment of the Redpanda Helm chart requires a separate namespace. Ensure you choose a unique namespace for each deployment. |

4.  Wait for the Redpanda cluster to be ready:

    ```bash
    kubectl --namespace <namespace> rollout status statefulset redpanda --watch
    ```

    When the Redpanda cluster is ready, the output should look similar to the following:

    statefulset rolling update complete 3 pods at revision redpanda-8654f645b4...

5.  Verify that each Redpanda broker is scheduled on only one Kubernetes node:

    ```bash
    kubectl get pod --namespace <namespace> \
    -o=custom-columns=NODE:.spec.nodeName,NAME:.metadata.name -l \
    app.kubernetes.io/component=redpanda-statefulset
    ```

    Expected output:

    example-worker3   redpanda-0
    example-worker2   redpanda-1
    example-worker    redpanda-2

## [](#deploy-multiple-redpanda-clusters)Deploy multiple Redpanda clusters

You can deploy multiple Redpanda clusters in the same Kubernetes cluster. This is useful for creating separate environments (such as production, staging, and development) or for organizing clusters by application or team.

### Operator

When using the Redpanda Operator, you can deploy multiple Redpanda clusters by creating separate Redpanda custom resources.

**Requirements:**

-   Use a cluster-scoped Redpanda Operator deployment (recommended) or separate namespace-scoped operators in different namespaces

-   Each cluster must be deployed in a unique namespace

-   Configure unique external port numbers for each cluster to avoid conflicts

    1.  Create a second Redpanda cluster in a different namespace:

        `redpanda-cluster-two.yaml`

        ```yaml
        apiVersion: cluster.redpanda.com/v1alpha2
        kind: Redpanda
        metadata:
          name: redpanda-staging
          namespace: redpanda-staging
        spec:
          clusterSpec:
            image:
              tag: v25.3.11
            listeners:
              kafka:
                external:
                  default:
                    advertisedPorts: [31093] (1)
              admin:
                external:
                  default:
                    advertisedPorts: [31645] (1)
              http:
                external:
                  default:
                    advertisedPorts: [30083] (1)
              rpc:
                port: 33146 (1)
              schemaRegistry:
                external:
                  default:
                    advertisedPorts: [30084] (1)
            statefulset:
              extraVolumes: |-
                - name: redpanda-io-config
                  configMap:
                    name: redpanda-io-config
              extraVolumeMounts: |-
                - name: redpanda-io-config
                  mountPath: /etc/redpanda-io-config
              additionalRedpandaCmdFlags:
                - "--io-properties-file=/etc/redpanda-io-config/io-config.yaml"
        ```

        | 1 | Configure unique port numbers for each cluster to avoid conflicts. Ensure these ports don’t conflict with your first cluster’s configuration. |
        | --- | --- |

    2.  Apply the second Redpanda resource:

        ```bash
        kubectl apply -f redpanda-cluster-two.yaml
        ```

    3.  Wait for the second cluster to be ready:

        ```bash
        kubectl get redpanda --namespace redpanda-staging --watch
        ```

### Helm

When using Helm, deploy multiple Redpanda clusters by using separate namespaces and unique release names for each deployment.

**Requirements:**

-   Each cluster must be deployed in a unique namespace

-   Use unique Helm release names for each deployment

-   Configure unique external port numbers for each cluster to avoid conflicts

    1.  Create configuration values for your second cluster:

        `redpanda-staging-values.yaml`

        ```yaml
        image:
          tag: v25.3.11
        nameOverride: 'redpanda-staging'
        fullnameOverride: 'redpanda-staging'
        listeners:
          kafka:
            external:
              default:
                advertisedPorts: [31093] (1)
          admin:
            external:
              default:
                advertisedPorts: [31645] (1)
          http:
            external:
              default:
                advertisedPorts: [30083] (1)
          rpc:
            port: 33146 (1)
          schemaRegistry:
            external:
              default:
                advertisedPorts: [30084] (1)
        statefulset:
          extraVolumes: |-
            - name: redpanda-io-config
              configMap:
                name: redpanda-io-config
          extraVolumeMounts: |-
            - name: redpanda-io-config
              mountPath: /etc/redpanda-io-config
          additionalRedpandaCmdFlags:
            - "--io-properties-file=/etc/redpanda-io-config/io-config.yaml"
        ```

        | 1 | Configure unique port numbers for each cluster to avoid conflicts. Ensure these ports don’t conflict with your first cluster’s configuration. |
        | --- | --- |

    2.  Install the second Redpanda cluster using a unique release name and namespace:

        ```bash
        helm install redpanda-staging redpanda/redpanda \
          --version 26.1.4 \
          --namespace redpanda-staging \
          --create-namespace \
          --values redpanda-staging-values.yaml
        ```

    3.  Wait for the second cluster to be ready:

        ```bash
        kubectl --namespace redpanda-staging rollout status statefulset redpanda-staging --watch
        ```

> ❗ **IMPORTANT**
>
> When deploying multiple clusters, ensure that external listener ports are unique across all clusters to prevent conflicts. Also consider resource allocation and node capacity when planning multiple cluster deployments.

## [](#config)Production considerations

This section provides advice for configuring the Redpanda in Kubernetes for production.

If you’re using the Redpanda Operator, see: [cluster.redpanda.com/v1alpha2](https://docs.redpanda.com/streaming/25.3/reference/k-crd/) for all available settings.

If you’re using the Redpanda Helm chart, see: [Redpanda Helm Chart Specification](https://docs.redpanda.com/streaming/25.3/reference/k-redpanda-helm-spec/) for all available settings.

### [](#version-pinning)Version pinning (Helm)

If you use the Redpanda Helm chart to deploy Redpanda, it’s important to pin the version of the Helm chart to ensure that you have control over the version of Redpanda that you deploy.

The Redpanda Helm chart version is independent of the Redpanda application version. The Redpanda application version can change even in patch releases of the Helm chart. This means that updates to the chart may roll out new versions of Redpanda.

To avoid unexpected changes to your deployments, pin the version of the Helm chart. Pinning refers to the practice of specifying an exact version to use during deployment, rather than using the latest or unspecified version. When you pin the Helm chart version, you maintain consistent, predictable environments, especially in production. Using a specific version helps to:

-   **Ensure compatibility**: Guarantee that the deployed application behaves as tested, regardless of new chart versions being released.

-   **Avoid unexpected updates**: Prevent automatic updates that may introduce changes incompatible with the current deployment or operational practices.


```bash
helm repo add redpanda https://charts.redpanda.com
helm repo update
helm install redpanda redpanda/redpanda \
  --version 26.1.4 \
  --namespace <namespace> \
  --create-namespace
```

[Review the release notes](https://docs.redpanda.com/streaming/25.3/reference/releases/) to understand any significant changes, bug fixes, or potential disruptions that could affect your existing deployment.

### [](#name-overrides-helm)Name overrides (Helm)

Deploying multiple instances of the same Helm chart in a Kubernetes cluster can lead to naming conflicts. Using `nameOverride` and `fullnameOverride` helps differentiate between them. If you have a production and staging environment for Redpanda, different names help to avoid confusion.

-   Use `nameOverride` to customize the labels `app.kubernetes.io/component=<nameOverride>-statefulset` and `app.kubernetes.io/name=<nameOverride>`.

-   Use `fullnameOverride` to customize the name of the StatefulSet and Services.


```yaml
nameOverride: 'redpanda-production'
fullnameOverride: 'redpanda-instance-prod'
```

### [](#labels)Labels

Kubernetes labels help you to organize, query, and manage your resources. Use labels to categorize Kubernetes resources in different deployments by environment, purpose, or team.

```yaml
commonLabels:
  env: 'production'
```

### [](#tolerations)Tolerations

Tolerations and taints allow Pods to be scheduled onto nodes where they otherwise wouldn’t. If you have nodes dedicated to Redpanda with a taint `dedicated=redpanda:NoSchedule`, the following toleration allows the Redpanda brokers to be scheduled on them.

```yaml
tolerations:
- key: "dedicated"
  operator: "Equal"
  value: "redpanda"
  effect: "NoSchedule"
```

### [](#docker-image)Docker image

You can specify the image tag to deploy a known version of the Redpanda Docker image. By default, the image tag is set in `Chart.appVersion`. Avoid using the `latest` tag, which can lead to unexpected changes.

If you’re using a private repository, always ensure your nodes have the necessary credentials to pull the image.

```yaml
image:
  repository: docker.redpanda.com/redpandadata/redpanda
  tag: "v25.3.11"
imagePullSecrets: []
```

### [](#number-of-redpanda-brokers)Number of Redpanda brokers

The number of Redpanda brokers you deploy depends on your use case and the level of redundancy you require. For production, deploy at least three Redpanda brokers. Always deploy an odd number of brokers to avoid split-brain scenarios.

```yaml
statefulset:
  replicas: 3
```

> 📝 **NOTE**
>
> You must provision one dedicated worker node for each Redpanda broker that you plan to deploy in your Redpanda cluster. The default [`podAntiAffinity` rules](#affinity-rules) make sure that each Redpanda broker runs on its own worker node.

See also:

-   [High Availability in Kubernetes](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-high-availability/)

-   [Kubernetes Cluster Requirements](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-requirements/#number-of-worker-nodes)


### [](#tls)TLS

By default, TLS (Transport Layer Security) is enabled for encrypted communication. Internal (`default`) and external (`external`) self-signed certificates are generated using cert-manager. See [TLS Certificates](#tls-certificates).

```yaml
tls:
  enabled: true
  certs:
    # This key represents the name of the certificate.
    default:
      caEnabled: true
    # This key represents the name of the certificate.
    external:
      caEnabled: true
```

See also: [TLS for Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/security/tls/)

### [](#authentication)Authentication

If you want to authenticate clients connections to the Redpanda cluster, you can enable SASL authentication.

```yaml
auth:
  sasl:
    enabled: true
    mechanism: "SCRAM-SHA-512"
    secretRef: "sasl-password-secret"
    users: []
```

See also: [Configure Authentication for Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/security/authentication/k-authentication/)

### [](#resources)Resources

By default, the resources allocated to Redpanda are for a development environment. In a production cluster, the resources you allocate should be proportionate to your machine type. You should determine and set these values before deploying the cluster.

```yaml
resources:
  cpu:
    cores: 4
  memory:
    enable_memory_locking: true
    container:
      max: 10Gi
```

See also:

-   [Manage Pod Resources in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/k-manage-resources/)

-   [Kubernetes Cluster Requirements and Recommendations](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-requirements/)


### [](#storage)Storage

In production, it’s best to use local PersistentVolumes (PVs) that are backed by NVMe devices to store the Redpanda data directory. NVMe devices outperform traditional SSDs or HDDs.

Redpanda Data recommends creating StorageClasses that use the [local volume manager (LVM) CSI driver](https://github.com/metal-stack/csi-driver-lvm) to automatically provision PVs. The LVM allows you to group physical storage devices into a logical volume group. Allocating logical volumes from a logical volume group provides greater flexibility in terms of storage expansion and management. The LVM supports features such as resizing, snapshots, and striping, which are not available with the other drivers such as the local volume static provisioner.

```yaml
storage:
  persistentVolume:
    enabled: true
    size: 100Gi
    storageClass: csi-driver-lvm-striped-xfs
```

For an example of configuring local PersistentVolumes backed by NVMe disks, see one of the following guides:

-   [Azure Kubernetes Service](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/aks-guide/#create-sc) (AKS)

-   [Elastic Kubernetes Service](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/eks-guide/#create-sc) (EKS)

-   [Google Kubernetes Engine](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/gke-guide/#create-sc) (GKE)


See also:

-   [Supported Volume Types for Data in Redpanda](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/storage/k-volume-types/)

-   [Kubernetes Cluster Requirements and Recommendations](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-requirements/)

-   [Configure Storage for the Redpanda data directory in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/storage/k-configure-storage/)


### [](#external-access)External access

To make the Redpanda cluster accessible from outside the Kubernetes cluster, you can use NodePort or LoadBalancer Services.

The default NodePort Service provides the lowest latency of all the Kubernetes Services because it does not include any unnecessary routing or middleware. Client connections go to the Redpanda brokers in the most direct way possible, through the worker nodes.

By default, the fully qualified domain names (FQDNs) that brokers advertise are their internal addresses within the Kubernetes cluster, which are not reachable from outside the cluster. To make the cluster accessible from outside, each broker must advertise a domain that can be reached from outside the cluster.

```yaml
external:
  enabled: true
  type: NodePort
```

See also:

-   [About Networking and Connectivity in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/networking/k-networking-and-connectivity/)

-   [Configure Listeners in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/networking/k-configure-listeners/)


### [](#externaldns)ExternalDNS

You should use ExternalDNS to manage DNS records for your Pods' domains. ExternalDNS synchronizes exposed Kubernetes Services with various DNS providers, rendering Kubernetes resources accessible through DNS servers.

Benefits of ExternalDNS include:

-   **Automation**: ExternalDNS automatically configures public DNS records when you create, update, or delete Kubernetes Services or Ingresses. This eliminates the need for manual DNS configuration, which can be error-prone.

-   **Compatibility**: ExternalDNS is compatible with a wide range of DNS providers, including major cloud providers such as AWS, Google Cloud, and Azure, and DNS servers like CoreDNS and PowerDNS.

-   **Integration with other tools**: ExternalDNS can be used with other Kubernetes tools, such as ingress controllers or cert-manager for managing TLS certificates.


```yaml
external:
  enabled: true
  type: LoadBalancer
  externalDns:
    enabled: true
```

See also:

-   [ExternalDNS with a NodePort Service](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/networking/external/k-nodeport/#externaldns)

-   [ExternalDNS with LoadBalancer Services](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/networking/external/k-loadbalancer/#externaldns)


### [](#logging)Logging

By default, the log-level is set to `info`. In production, use the `info` logging level to avoid overwhelming the storage. For debugging purposes, temporarily change the logging level to `debug`.

```yaml
logging:
  level: "info"
```

### [](#monitoring)Monitoring

By default, monitoring is disabled. If you have the [Prometheus Operator](https://prometheus-operator.dev/), enable monitoring to deploy a ServiceMonitor resource for Redpanda. Observability is essential in production environments.

```yaml
monitoring:
  enabled: true
```

See also: [Monitor in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/monitoring/)

### [](#statefulset-update-strategy)StatefulSet update strategy

For smooth and uninterrupted updates, use the default `RollingUpdate` strategy. Additionally, set a PodDisruptionBudget to ensure that at least one Pod is available during updates.

```yaml
statefulset:
  updateStrategy:
    type: "RollingUpdate"
  budget:
    maxUnavailable: 1
```

See also: [Upgrade Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/upgrade/k-rolling-upgrade/)

### [](#affinity-rules)Affinity rules

By default, `podAntiAffinity` rules stop the Kubernetes scheduler from placing multiple Redpanda brokers on the same node. These rules offer two benefits:

-   Minimize the risk of data loss by ensuring that a node’s failure results in the loss of only one Redpanda broker.

-   Prevent resource contention between brokers by ensuring they are never co-located on the same node.


Affinities control Pod placement in the cluster based on various conditions. Set these according to your high availability and infrastructure needs. For example, this is a soft rule that tries to ensure the Kubernetes scheduler doesn’t place two Pods with the same `app: redpanda` label in the same zone. However, if it’s not possible, the scheduler can still place the Pods in the same zone.

```yaml
statefulset:
  podAntiAffinity:
    topologyKey: kubernetes.io/hostname
    type: hard
    weight: 100
    custom:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: "app"
              operator: "In"
              values:
              - "redpanda"
          topologyKey: "kubernetes.io/zone"
```

To completely disable `podAntiAffinity` rules (for example, in development environments where you want to run multiple brokers on the same node), you can override the default configuration:

```yaml
statefulset:
  podTemplate:
    spec:
      affinity:
        podAntiAffinity: null
```

> ⚠️ **WARNING**
>
> Disabling `podAntiAffinity` rules is not recommended for production environments as it allows multiple brokers to be scheduled on the same node, increasing the risk of data loss if a node fails.

See also: [High Availability in Kubernetes](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-high-availability/)

### [](#graceful-shutdown)Graceful shutdown

By default, Pods are given 90 seconds to shut down gracefully. If your brokers require additional time for a graceful shutdown, modify the `terminationGracePeriodSeconds`.

```yaml
statefulset:
  terminationGracePeriodSeconds: 100
```

See also: [Upgrade Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/upgrade/k-rolling-upgrade/)

### [](#service-account)Service account

Restricting permissions is a best practice. Create a dedicated ServiceAccount for each Pod. To assign roles to this ServiceAccount, see [Role-based access control (RBAC)](#RBAC).

```yaml
serviceAccount:
  create: true
  name: "redpanda-service-account"
```

### [](#RBAC)Role-based access control (RBAC)

RBAC is a method for providing permissions to ServiceAccounts based on roles. Some features such as rack awareness require both a ServiceAccount and RBAC to access resources using the Kubernetes API.

```yaml
rbac:
  enabled: true
  annotations: {}
```

See also: [Enable Rack Awareness in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/k-rack-awareness/)

## [](#perform-a-self-test)Perform a self test

To understand the performance capabilities of your Redpanda cluster, Redpanda offers built-in self-test features that evaluate the performance of both disk and network operations.

For more information, see [Disk and network self-test benchmarks](https://docs.redpanda.com/streaming/25.3/troubleshoot/cluster-diagnostics/diagnose-issues/#self-test).

When using the storage bandwidth test, ensure that your results show at least 16,000 IOPS (Input/Output Operations Per Second) for production environments. If your test results are below this threshold, your storage may not be suitable for production Redpanda workloads.

## [](#redpanda-console)Redpanda Console

Redpanda Console is deployed by default when you deploy a Redpanda cluster using either the Redpanda Operator or the Redpanda Helm chart. This provides a web UI for managing and debugging your Redpanda cluster and applications.

Both the Redpanda Operator and Helm chart automatically deploy Redpanda Console alongside your Redpanda cluster with sensible defaults.

If you need to deploy Redpanda Console separately (for example, to connect to a Redpanda cluster running outside Kubernetes, or to have more granular control over the configuration), see [Deploy Redpanda Console on Kubernetes](https://docs.redpanda.com/streaming/25.3/deploy/console/kubernetes/deploy/).

The standalone deployment option provides the option to connect to Redpanda clusters running outside Kubernetes.

For resource recommendations and scaling guidance for Redpanda Console on Kubernetes, see [Redpanda Console Kubernetes Requirements and Recommendations](https://docs.redpanda.com/streaming/25.3/deploy/console/kubernetes/k-requirements/).

## [](#explore-the-default-kubernetes-components)Explore the default Kubernetes components

By default, the Redpanda Helm chart deploys the following Kubernetes components:

-   [A StatefulSet](#statefulset) with three Pods.

-   [One PersistentVolumeClaim](#persistentvolumeclaim) for each Pod, each with a capacity of 20Gi.

-   [A headless ClusterIP Service and a NodePort Service](#service) for each Kubernetes node that runs a Redpanda broker.

-   [Self-Signed TLS Certificates](#tls-certificates).


### [](#statefulset)StatefulSet

Redpanda is a stateful application. Each Redpanda broker needs to store its own state (topic partitions) in its own storage volume. As a result, the Helm chart deploys a StatefulSet to manage the Pods in which the Redpanda brokers are running.

```bash
kubectl get statefulset --namespace <namespace>
```

Example output:

NAME       READY   AGE
redpanda   3/3     3m11s

StatefulSets ensure that the state associated with a particular Pod replica is always the same, no matter how often the Pod is recreated. Each Pod is also given a unique ordinal number in its name such as `redpanda-0`. A Pod with a particular ordinal number is always associated with a PersistentVolumeClaim with the same number. When a Pod in the StatefulSet is deleted and recreated, it is given the same ordinal number and so it mounts the same storage volume as the deleted Pod that it replaced.

```bash
kubectl get pod --namespace <namespace>
```

Expected output:

```none
NAME                              READY   STATUS      RESTARTS        AGE
redpanda-0                        1/1     Running     0               6m9s
redpanda-1                        1/1     Running     0               6m9s
redpanda-2                        1/1     Running     0               6m9s
redpanda-console-5ff45cdb9b-6z2vs 1/1     Running     0               5m
redpanda-configuration-smqv7      0/1     Completed   0               6m9s
```

> 📝 **NOTE**
>
> The `redpanda-configuration` job updates the Redpanda runtime configuration.

### [](#persistentvolumeclaim)PersistentVolumeClaim

Redpanda brokers must be able to store their data on disk. By default, the Helm chart uses the default StorageClass in the Kubernetes cluster to create a PersistentVolumeClaim for each Pod. The default StorageClass in your Kubernetes cluster depends on the Kubernetes platform that you are using.

```bash
kubectl get persistentvolumeclaims --namespace <namespace>
```

Expected output:

```none
NAME                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
datadir-redpanda-0   Bound    pvc-3311ade3-de84-4027-80c6-3d8347302962   20Gi       RWO            standard       75s
datadir-redpanda-1   Bound    pvc-4ea8bc03-89a6-41e4-b985-99f074995f08   20Gi       RWO            standard       75s
datadir-redpanda-2   Bound    pvc-45c3555f-43bc-48c2-b209-c284c8091c45   20Gi       RWO            standard       75s
```

### [](#service)Service

The clients writing to or reading from a given partition have to connect directly to the leader broker that hosts the partition. As a result, clients need to be able to connect directly to each Pod. To allow internal and external clients to connect to each Pod that hosts a Redpanda broker, the Helm chart configures two Services:

-   Internal using the [Headless ClusterIP](#headless-clusterip-service)

-   External using the [NodePort](#nodeport-service)


```bash
kubectl get service --namespace <namespace>
```

Expected output:

```none
NAME                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                       AGE
redpanda            ClusterIP   None            <none>        <none>                                                        5m37s
redpanda-console    ClusterIP   10.0.251.204    <none>        8080                                                          5m
redpanda-external   NodePort    10.96.137.220   <none>        9644:31644/TCP,9094:31092/TCP,8083:30082/TCP,8080:30081/TCP   5m37s
```

#### [](#headless-clusterip-service)Headless ClusterIP Service

The headless Service associated with a StatefulSet gives the Pods their network identity in the form of a fully qualified domain name (FQDN). Both Redpanda brokers in the same Redpanda cluster and clients within the same Kubernetes cluster use this FQDN to communicate with each other.

An important requirement of distributed applications such as Redpanda is peer discovery: The ability for each broker to find other brokers in the same cluster. When each Pod is rolled out, its `seed_servers` field is updated with the FQDN of each Pod in the cluster so that they can discover each other.

```bash
kubectl --namespace <namespace> exec redpanda-0 -c redpanda -- cat etc/redpanda/redpanda.yaml
```

```yaml
redpanda:
  data_directory: /var/lib/redpanda/data
  empty_seed_starts_cluster: false
  seed_servers:
  - host:
      address: redpanda-0.redpanda.<namespace>.svc.cluster.local.
      port: 33145
  - host:
      address: redpanda-1.redpanda.<namespace>.svc.cluster.local.
      port: 33145
  - host:
      address: redpanda-2.redpanda.<namespace>.svc.cluster.local.
      port: 33145
```

#### [](#nodeport-service)NodePort Service

External access is made available by a NodePort service that opens the following ports by default:

| Listener | Node Port | Container Port |
| --- | --- | --- |
| Schema Registry | 30081 | 8081 |
| HTTP Proxy | 30082 | 8083 |
| Kafka API | 31092 | 9094 |
| Admin API | 31644 | 9644 |

To learn more, see [Networking and Connectivity in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/networking/k-networking-and-connectivity/).

### [](#tls-certificates)TLS Certificates

By default, TLS is enabled in the Redpanda Helm chart. The Helm chart uses [cert-manager](https://cert-manager.io/docs/) to generate four Certificate resources that provide Redpanda with self-signed certificates for internal and external connections.

Having separate certificates for internal and external connections provides security isolation. If an external certificate or its corresponding private key is compromised, it doesn’t affect the security of internal communications.

```bash
kubectl get certificate --namespace <namespace>
```

NAME                                 READY
redpanda-default-cert                True
redpanda-default-root-certificate    True
redpanda-external-cert               True
redpanda-external-root-certificate   True

-   `redpanda-default-cert`: Self-signed certificate for internal communications.

-   `redpanda-default-root-certificate`: Root certificate authority for the internal certificate.

-   `redpanda-external-cert`: Self-signed certificate for external communications.

-   `redpanda-external-root-certificate`: Root certificate authority for the external certificate.


By default, all listeners are configured with the same certificate. To configure separate TLS certificates for different listeners, see [TLS for Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/security/tls/).

> 📝 **NOTE**
>
> The Redpanda Helm chart provides self-signed certificates for convenience. In a production environment, it’s best to use certificates from a trusted Certificate Authority (CA) or integrate with your existing CA infrastructure.

## [](#uninstall-redpanda)Uninstall Redpanda

When you finish testing Redpanda, you can uninstall it from your Kubernetes cluster. The steps depend on how you installed Redpanda: using the Redpanda Operator or the Redpanda Helm chart.

### Operator

Follow the steps in **exact order** to avoid race conditions between the Redpanda Operator’s reconciliation loop and Kubernetes garbage collection.

1.  Delete all Redpanda-related custom resources:

    ```bash
    kubectl delete users      --namespace <namespace> --all
    kubectl delete topics     --namespace <namespace> --all
    kubectl delete schemas    --namespace <namespace> --all
    kubectl delete redpanda   --namespace <namespace> --all
    ```

2.  Make sure requests for those resources return no results. For example, if you had a Redpanda cluster named `redpanda` in the namespace `<namespace>`, run:

    ```bash
    kubectl get redpanda --namespace <namespace>
    ```

3.  Uninstall the Redpanda Operator Helm release:

    ```bash
    helm uninstall redpanda-controller --namespace <namespace>
    ```

    Helm does not uninstall CRDs by default when using `helm uninstall` to avoid accidentally deleting existing custom resources.

4.  Remove the CRDs.

    1.  List all Redpanda CRDs installed by the operator:

        ```bash
        kubectl api-resources --api-group='cluster.redpanda.com'
        ```

        This command displays all CRDs defined by the Redpanda Operator. For example:

        ```bash
        NAME        SHORTNAMES   APIVERSION                      NAMESPACED   KIND
        redpandas   rp           cluster.redpanda.com/v1alpha2   true         Redpanda
        schemas     sc           cluster.redpanda.com/v1alpha2   true         Schema
        topics                   cluster.redpanda.com/v1alpha2   true         Topic
        users       rpu          cluster.redpanda.com/v1alpha2   true         User
        ```

    2.  Delete the CRDs:

        ```bash
        kubectl get crds -o name | grep cluster.redpanda.com | xargs kubectl delete
        ```

        This command lists all CRDs with the `cluster.redpanda.com` domain suffix and deletes them, ensuring only Redpanda CRDs are removed. Helm does not delete CRDs automatically to prevent data loss, so you must run this step manually.


5.  (Optional) Delete any leftover PVCs or Secrets in the namespace:

    > ⚠️ **CAUTION**
    >
    > The following command deletes all PVCs and Secrets in the namespace, which may remove unrelated resources if the namespace is shared with other applications.

    ```bash
    kubectl delete pvc,secret --all --namespace <namespace>
    ```

### Helm

If you deployed Redpanda with the Redpanda Helm chart, follow these steps to uninstall it:

1.  Uninstall the Helm release:

    ```bash
    helm uninstall redpanda --namespace <namespace>
    ```

2.  (Optional) Delete any leftover PVCs or Secrets in the namespace:

    > ⚠️ **CAUTION**
    >
    > The following command deletes all PVCs and Secrets in the namespace, which may remove unrelated resources if the namespace is shared with other applications.

    ```bash
    kubectl delete pvc,secret --all --namespace <namespace>
    ```

## [](#troubleshoot)Troubleshoot

Before troubleshooting your cluster, make sure that you have all the [prerequisites](#prerequisites).

### [](#helm-v3-18-0-is-not-supported-json-number-error)Helm v3.18.0 is not supported (json.Number error)

If you are using Helm v3.18.0, you may encounter errors such as:

Error: INSTALLATION FAILED: execution error at (redpanda/templates/entry-point.yaml:17:4): invalid Quantity expected string or float64 got: json.Number (1)

This is due to a bug in Helm v3.18.0. To avoid similar errors, upgrade to a later version. For more details, see the [Helm GitHub issue](https://github.com/helm/helm/issues/30880). === StatefulSet never rolls out

If the StatefulSet Pods remain in a pending state, they are waiting for resources to become available.

To identify the Pods that are pending, use the following command:

```bash
kubectl get pod --namespace <namespace>
```

The response includes a list of Pods in the StatefulSet and their status.

To view logs for a specific Pod, use the following command.

```bash
kubectl logs -f <pod-name> --namespace <namespace>
```

You can use the output to debug your deployment.

### [](#didnt-match-pod-anti-affinity-rules)Didn’t match pod anti-affinity rules

If you see this error, your cluster does not have enough nodes to satisfy the anti-affinity rules:

Warning  FailedScheduling  18m  default-scheduler  0/1 nodes are available: 1 node(s) didn't match pod anti-affinity rules. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.

The Helm chart configures default `podAntiAffinity` rules to make sure that only one Pod running a Redpanda broker is scheduled on each worker node. To learn why, see [Number of workers](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-requirements/#number-of-workers).

To resolve this issue, do one of the following:

-   Create additional worker nodes.

-   Modify the anti-affinity rules (for development purposes only).

    If adding nodes is not an option, you can modify the `podAntiAffinity` rules in your StatefulSet to be less strict.

    #### Operator

    `redpanda-cluster.yaml`

    ```yaml
    apiVersion: cluster.redpanda.com/v1alpha2
    kind: Redpanda
    metadata:
      name: redpanda
    spec:
      chartRef: {}
      clusterSpec:
        statefulset:
          podAntiAffinity:
            type: soft
    ```

    ```bash
    kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
    ```


    #### Helm


    ##### --values

    `docker-repo.yaml`

    ```yaml
    statefulset:
      podAntiAffinity:
        type: soft
    ```

    ```bash
    helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
      --values docker-repo.yaml
    ```


    ##### --set

    ```bash
    helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
      --set statefulset.podAntiAffinity.type=soft
    ```


### [](#unable-to-mount-volume)Unable to mount volume

If you see volume mounting errors in the Pod events or in the Redpanda logs, ensure that each of your Pods has a volume available in which to store data.

-   If you’re using StorageClasses with dynamic provisioners (default), ensure they exist:

    ```bash
    kubectl get storageclass
    ```

-   If you’re using PersistentVolumes, ensure that you have one PersistentVolume available for each Redpanda broker, and that each one has the storage capacity that’s set in `storage.persistentVolume.size`:

    ```bash
    kubectl get persistentvolume --namespace <namespace>
    ```


To learn how to configure different storage volumes, see [Configure Storage](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/storage/k-configure-storage/).

### [](#failed-to-pull-image)Failed to pull image

When deploying the Redpanda Helm chart, you may encounter Docker rate limit issues because the default registry URL is not recognized as a Docker Hub URL. The domain `docker.redpanda.com` is used for statistical purposes, such as tracking the number of downloads. It mirrors Docker Hub’s content while providing specific analytics for Redpanda.

Failed to pull image "docker.redpanda.com/redpandadata/redpanda:v<version>": rpc error: code = Unknown desc = failed to pull and unpack image "docker.redpanda.com/redpandadata/redpanda:v<version>": failed to copy: httpReadSeeker: failed open: unexpected status code 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

To fix this error, do one of the following:

-   Replace the `image.repository` value in the Helm chart with `docker.io/redpandadata/redpanda`. Switching to Docker Hub avoids the rate limit issues associated with `docker.redpanda.com`.

    #### Operator

    `redpanda-cluster.yaml`

    ```yaml
    apiVersion: cluster.redpanda.com/v1alpha2
    kind: Redpanda
    metadata:
      name: redpanda
    spec:
      chartRef: {}
      clusterSpec:
        image:
          repository: docker.io/redpandadata/redpanda
    ```

    ```bash
    kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
    ```


    #### Helm


    ##### --values

    `docker-repo.yaml`

    ```yaml
    image:
      repository: docker.io/redpandadata/redpanda
    ```

    ```bash
    helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
      --values docker-repo.yaml
    ```


    ##### --set

    ```bash
    helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
      --set image.repository=docker.io/redpandadata/redpanda
    ```

-   Authenticate to Docker Hub by logging in with your Docker Hub credentials. The `docker.redpanda.com` site acts as a reflector for Docker Hub. As a result, when you log in with your Docker Hub credentials, you will bypass the rate limit issues.


### [](#dig-not-defined)Dig not defined

This error means that you are using an unsupported version of [Helm](https://helm.sh/docs/intro/install/):

Error: parse error at (redpanda/templates/statefulset.yaml:203): function "dig" not defined

To fix this error, ensure that you are using the minimum required version: 3.10.0.

```bash
helm version
```

### [](#repository-name-already-exists)Repository name already exists

If you see this error, remove the `redpanda` chart repository, then try installing it again.

```bash
helm repo remove redpanda
helm repo add redpanda https://charts.redpanda.com
helm repo update
```

### [](#fatal-error-during-checker-data-directory-is-writable-execution)Fatal error during checker "Data directory is writable" execution

This error appears when Redpanda does not have write access to your configured storage volume under `storage` in the Helm chart.

Error: fatal error during checker "Data directory is writable" execution: open /var/lib/redpanda/data/test\_file: permission denied

To fix this error, set `statefulset.initContainers.setDataDirOwnership.enabled` to `true` so that the initContainer can set the correct permissions on the data directories.

### [](#cannot-patch-redpanda-with-kind-statefulset)Cannot patch "redpanda" with kind StatefulSet

This error appears when you run `helm upgrade` with the `--values` flag but do not include all your previous overrides.

Error: UPGRADE FAILED: cannot patch "redpanda" with kind StatefulSet: StatefulSet.apps "redpanda" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden

To fix this error, include all the value overrides from the previous installation using either the `--set` or the `--values` flags.

> ⚠️ **WARNING**
>
> Do not use the `--reuse-values` flag to upgrade from one version of the Helm chart to another. This flag stops Helm from using any new values in the upgraded chart.

### [](#cannot-patch-redpanda-console-with-kind-deployment)Cannot patch "redpanda-console" with kind Deployment

This error appears if you try to upgrade your deployment and you already have `console.enabled` set to `true`.

Error: UPGRADE FAILED: cannot patch "redpanda-console" with kind Deployment: Deployment.apps "redpanda-console" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map\[string\]string{"app.kubernetes.io/instance":"redpanda", "app.kubernetes.io/name":"console"}, MatchExpressions:\[\]v1.LabelSelectorRequirement(nil)}: field is immutable

To fix this error, set `console.enabled` to `false` so that Helm doesn’t try to deploy Redpanda Console again.

### [](#helm-is-in-a-pending-rollback-state)Helm is in a pending-rollback state

An interrupted Helm upgrade process can leave your Helm release in a `pending-rollback` state. This state prevents further actions like upgrades, rollbacks, or deletions through standard Helm commands. To fix this:

1.  Identify the Helm release that’s in a `pending-rollback` state:

    ```bash
    helm list --namespace <namespace> --all
    ```

    Look for releases with a status of `pending-rollback`. These are the ones that need intervention.

2.  Verify the Secret’s status to avoid affecting the wrong resource:

    ```bash
    kubectl --namespace <namespace> get secret --show-labels
    ```

    Identify the Secret associated with your Helm release by its `pending-rollback` status in the labels.

    > ⚠️ **WARNING**
    >
    > Ensure you have correctly identified the Secret to avoid unintended consequences. Deleting the wrong Secret could impact other deployments or services.

3.  Delete the Secret to clear the `pending-rollback` state:

    ```bash
    kubectl --namespace <namespace> delete secret -l status=pending-rollback
    ```


After clearing the `pending-rollback` state:

-   **Retry the upgrade**: Restart the upgrade process. You should investigate the initial failure to avoid getting into the `pending-rollback` state again.

-   **Perform a rollback**: If you need to roll back to a previous release, use `helm rollback <release-name> <revision>` to revert to a specific, stable release version.


### [](#crash-loop-backoffs)Crash loop backoffs

If a broker crashes after startup, or gets stuck in a crash loop, it can accumulate an increasing amount of stored state. This accumulated state not only consumes additional disk space but also prolongs the time required for each subsequent restart to process it.

To prevent infinite crash loops, the Redpanda Helm chart sets the [`crash_loop_limit`](https://docs.redpanda.com/streaming/25.3/reference/properties/broker-properties/#crash_loop_limit) broker configuration property to `5`. The crash loop limit is the number of consecutive crashes that can happen within one hour of each other. By default, the broker terminates immediately after hitting the `crash_loop_limit`. The Pod running Redpanda remains in a `CrashLoopBackoff` state until its internal consecutive crash counter is reset to zero.

To facilitate debugging in environments where a broker is stuck in a crash loop, you can also set the [`crash_loop_sleep_sec`](https://docs.redpanda.com/streaming/25.3/reference/properties/broker-properties/#crash_loop_sleep_sec) broker configuration property. This setting determines how long the broker sleeps before terminating the process after reaching the crash loop limit. By providing a window during which the Pod remains available, you can SSH into it and troubleshoot the issue.

Example configuration:

```yaml
config:
  node:
    crash_loop_limit: 5
    crash_loop_sleep_sec: 60
```

In this example, when the broker hits the `crash_loop_limit` of 5, it will sleep for 60 seconds before terminating the process. This delay allows administrators to access the Pod and troubleshoot.

To troubleshoot a crash loop backoff:

1.  Check the Redpanda logs from the most recent crashes:

    ```bash
    kubectl logs <pod-name> --namespace <namespace>
    ```

    > 📝 **NOTE**
    >
    > Kubernetes retains logs only for the current and the previous instance of a container. This limitation makes it difficult to access logs from earlier crashes, which may contain vital clues about the root cause of the issue. Given these log retention limitations, setting up a centralized logging system is crucial. Systems such as [Loki](https://grafana.com/docs/loki/latest/) or [Datadog](https://www.datadoghq.com/product/log-management/) can capture and store logs from all containers, ensuring you have access to historical data.

2.  Resolve the issue that led to the crash loop backoff.

3.  Reset the crash counter to zero to allow Redpanda to restart. You can do any of the following to reset the counter:

    -   Make changes to any of the following sections in the Redpanda Helm chart to trigger an update:

        -   `config.node`

        -   `config.tunable`


        For example:

        ```yaml
        config:
          node:
            crash_loop_limit: <new-integer>
        ```

    -   Delete the `startup_log` file in the broker’s data directory.

        ```bash
        kubectl exec <pod-name> --namespace <namespace> -- rm /var/lib/redpanda/data/startup_log
        ```

        > 📝 **NOTE**
        >
        > It might be challenging to execute this command within a Pod that is in a `CrashLoopBackoff` state due to the limited time during which the Pod is available before it restarts. Wrapping the command in a loop might work.

    -   Wait one hour since the last crash. The crash counter resets after one hour.



To avoid future crash loop backoffs and manage the accumulation of small segments effectively:

-   [Monitor](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/monitoring/k-monitor-redpanda/) the size and number of segments regularly.

-   Optimize your Redpanda configuration for segment management.

-   Consider implementing [Tiered Storage](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/tiered-storage/k-tiered-storage/) to manage data more efficiently.


### [](#a-redpanda-enterprise-edition-license-is-required)A Redpanda Enterprise Edition license is required

During a Redpanda upgrade, if enterprise features are enabled and a valid Enterprise Edition license is missing, Redpanda logs a warning and aborts the upgrade process on the first broker. This issue prevents a successful upgrade.

A Redpanda Enterprise Edition license is required to use the currently enabled features. To apply your license, downgrade this broker to the pre-upgrade version and provide a valid license key via rpk using 'rpk cluster license set <key>', or via Redpanda Console. To request an enterprise license, please visit <redpanda.com/upgrade>. To try Redpanda Enterprise for 30 days, visit <redpanda.com/try-enterprise>. For more information, see <https://docs.redpanda.com/current/get-started/licenses>.

If you encounter this message, follow these steps to recover:

1.  [Roll back the affected broker to the original version](https://docs.redpanda.com/streaming/25.3/upgrade/k-rolling-upgrade/#roll-back).

2.  Do one of the following:

    -   [Apply a valid Redpanda Enterprise Edition license](https://docs.redpanda.com/streaming/25.3/get-started/licensing/add-license-redpanda/) to the cluster.

    -   Disable enterprise features.

        If you do not have a valid license and want to proceed without using enterprise features, you can disable the enterprise features in your Redpanda configuration.


3.  Retry the upgrade.


For more troubleshooting steps, see [Troubleshoot Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/troubleshoot/errors-solutions/k-resolve-errors/).

## [](#next-steps)Next steps

After deploying Redpanda, validate your production readiness:

-   [Production readiness checklist](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-production-readiness/) - Comprehensive validation of your deployment against production standards


See the [Manage Kubernetes topics](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/) to learn how to customize your deployment to meet your needs.

## [](#suggested-reading)Suggested reading

-   [High Availability in Kubernetes](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-high-availability/)

-   [Redpanda Helm Specification](https://docs.redpanda.com/streaming/25.3/reference/k-redpanda-helm-spec/)

-   [Redpanda CRD Reference](https://docs.redpanda.com/streaming/25.3/reference/k-crd/)


## Suggested labs

-   [Set Up GitOps for the Redpanda Helm Chart](https://docs.redpanda.com/labs/kubernetes/gitops-helm/)

[Search all labs](https://docs.redpanda.com/labs)