# Deploy a Local Development Cluster with kind or minikube > For the complete documentation index, see [llms.txt](https://docs.redpanda.com/llms.txt). Component-specific: [streaming-full.txt](https://docs.redpanda.com/streaming-full.txt) --- title: Deploy a Local Development Cluster with kind or minikube latest-redpanda-tag: v25.3.11 latest-console-tag: v3.7.3 latest-operator-version: v26.1.4 # EOL = End-of-Life (support lifecycle status) page-is-nearing-eol: "false" page-is-past-eol: "false" page-eol-date: November 19, 2026 latest-connect-version: 4.93.0 docname: redpanda/kubernetes/local-guide page-component-name: streaming page-version: "25.3" page-component-version: "25.3" page-component-title: Streaming page-relative-src-path: redpanda/kubernetes/local-guide.adoc page-edit-url: https://github.com/redpanda-data/docs/edit/v/25.3/modules/deploy/pages/redpanda/kubernetes/local-guide.adoc description: Deploy a local Redpanda cluster with Redpanda Console using the Helm chart. page-git-created-date: "2025-08-15" page-git-modified-date: "2025-08-15" support-status: supported --- Deploy a local Redpanda cluster with Redpanda Console using the Helm chart. Explore the essentials of how Redpanda works in Kubernetes and what components are deployed by default. Then, use `rpk` both as an internal client and an external client to interact with your Redpanda cluster from the command line. > ⚠️ **CAUTION: Only for development and testing** > > Only for development and testing > > Do not use kind or minikube for production workloads. Instead, try one of the following environments: > > - [Azure Kubernetes Service](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/aks-guide/) (AKS) > > - [Elastic Kubernetes Service](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/eks-guide/) (EKS) > > - [Google Kubernetes Engine](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/gke-guide/) (GKE) ## [](#prerequisites)Prerequisites Before you begin, make sure that you have the correct software for your Kubernetes platform: ### kind - [Install `kubectl`](https://kubernetes.io/docs/tasks/tools/). Minimum required Kubernetes version: 1.27.0-0 ```bash kubectl version --client ``` - [Install Helm](https://helm.sh/docs/intro/install/). Minimum required Helm version: 3.10.0 ```bash helm version ``` - [Install kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) - [Install Docker](https://docs.docker.com/get-docker/) ### minikube - [Install `kubectl`](https://kubernetes.io/docs/tasks/tools/). Minimum required Kubernetes version: 1.27.0-0 ```bash kubectl version --client ``` - [Install Helm](https://helm.sh/docs/intro/install/). Minimum required Helm version: 3.10.0 ```bash helm version ``` - [Install minikube](https://minikube.sigs.k8s.io/docs/start/) ## [](#create-a-kubernetes-cluster)Create a Kubernetes cluster In this step, you create one master and three worker nodes (one worker node for each Redpanda broker). ### kind 1. Define a cluster in the `kind.yaml` configuration file: ```bash cat <kind.yaml --- apiVersion: kind.x-k8s.io/v1alpha4 kind: Cluster nodes: - role: control-plane - role: worker - role: worker - role: worker EOF ``` 2. Create the Kubernetes cluster from the configuration file: ```bash kind create cluster --config kind.yaml ``` ### minikube 1. Create the Kubernetes cluster: ```bash minikube start --nodes 4 ``` 2. Prevent applications from being scheduled on the Kubernetes control plane node: ```bash kubectl taint node \ -l node-role.kubernetes.io/control-plane="" \ node-role.kubernetes.io/control-plane=:NoSchedule ``` > 📝 **NOTE** > > The Helm chart configures default `podAntiAffinity` rules to make sure that only one Pod running a Redpanda broker is scheduled on each worker node. To learn why, see [Number of workers](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-requirements/#number-of-workers). ## [](#deploy-redpanda-and-redpanda-console)Deploy Redpanda and Redpanda Console In this step, you deploy Redpanda with self-signed TLS certificates. Redpanda Console is included as a subchart in the Redpanda Helm chart. ### Operator 1. Make sure that you have permission to install custom resource definitions (CRDs): ```bash kubectl auth can-i create CustomResourceDefinition --all-namespaces ``` You should see `yes` in the output. You need these cluster-level permissions to install [cert-manager](https://cert-manager.io/docs/) and Redpanda Operator CRDs in the next steps. 2. Install [cert-manager](https://cert-manager.io/docs/installation/helm/) using Helm: ```bash helm repo add jetstack https://charts.jetstack.io helm repo update helm install cert-manager jetstack/cert-manager --set crds.enabled=true --namespace cert-manager --create-namespace ``` TLS is enabled by default. The Redpanda Helm chart uses cert-manager to manage TLS certificates by default. 3. Deploy the Redpanda Operator: 1. To deploy in cluster scope, use: ```bash helm repo add redpanda https://charts.redpanda.com helm repo update helm upgrade --install redpanda-controller redpanda/operator \ --namespace \ --create-namespace \ --version v26.1.4 \ (1) --set crds.enabled=true (2) ``` | 1 | This flag specifies the exact version of the Redpanda Operator Helm chart to use for deployment. By setting this value, you pin the chart to a specific version, which prevents automatic updates that might introduce breaking changes or new features that have not been tested in your environment. | | --- | --- | | 2 | This flag ensures that the CRDs are installed as part of the Redpanda Operator deployment.This command deploys the Redpanda Operator in cluster scope (default in v25.2+), allowing it to manage Redpanda clusters across multiple namespaces. | 2. To deploy in namespace scope (managing only resources within its deployment namespace), use: ```bash helm upgrade --install redpanda-controller redpanda/operator \ --namespace \ --create-namespace \ --version v26.1.4 \ --set crds.enabled=true \ --set 'additionalCmdFlags=["--namespace="]' (1) ``` | 1 | This flag restricts the Redpanda Operator to manage resources only within the specified namespace. | | --- | --- | 4. Ensure that the Deployment is successfully rolled out: ```bash kubectl --namespace rollout status --watch deployment/redpanda-controller-operator ``` deployment "redpanda-controller-operator" successfully rolled out 5. Install a [Redpanda custom resource](https://docs.redpanda.com/streaming/25.3/reference/k-crd/) in the same namespace as the Redpanda Operator: `redpanda-cluster.yaml` ```yaml apiVersion: cluster.redpanda.com/v1alpha2 kind: Redpanda metadata: name: redpanda spec: clusterSpec: external: domain: customredpandadomain.local statefulset: initContainers: setDataDirOwnership: enabled: true ``` ```bash kubectl apply -f redpanda-cluster.yaml --namespace ``` 6. Wait for the Redpanda Operator to deploy Redpanda using the Helm chart: ```bash kubectl get redpanda --namespace --watch ``` NAME READY STATUS redpanda True Redpanda reconciliation succeeded This step may take a few minutes. You can watch for new Pods to make sure that the deployment is progressing: ```bash kubectl get pod --namespace ``` If it’s taking too long, see [Troubleshoot](#troubleshoot). ### Helm 1. Add the Redpanda Helm chart repository and install cert-manager using Helm: ```bash helm repo add redpanda https://charts.redpanda.com helm repo add jetstack https://charts.jetstack.io helm repo update helm install cert-manager jetstack/cert-manager --set crds.enabled=true --namespace cert-manager --create-namespace ``` The Redpanda Helm chart uses cert-manager to manage TLS certificates. 2. Install Redpanda using Helm: ```bash helm repo add redpanda https://charts.redpanda.com/ helm repo update helm install redpanda redpanda/redpanda \ --version 26.1.4 \ --namespace \ --create-namespace \ --set external.domain=customredpandadomain.local \ --set statefulset.initContainers.setDataDirOwnership.enabled=true ``` The installation displays some tips for getting started. 3. Wait for the Redpanda cluster to be ready: ```bash kubectl --namespace rollout status statefulset redpanda --watch ``` When the Redpanda cluster is ready, the output should look similar to the following: ```plain statefulset rolling update complete 3 pods at revision redpanda-8654f645b4... ``` If your cluster remains in a pending state, see [Troubleshoot](#troubleshoot). ## [](#start-streaming)Start streaming Each Redpanda broker comes with `rpk`, which is a CLI tool for connecting to and interacting with Redpanda brokers. You can use `rpk` inside one of the Redpanda broker’s Docker containers to create a topic, produce messages to it, and consume messages from it. 1. Create an alias to simplify the `rpk` commands: ```bash alias internal-rpk="kubectl --namespace exec -i -t redpanda-0 -c redpanda -- rpk" ``` 2. Create a topic called `twitch-chat`: ### Operator 1. Create a [Topic resource](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/k-manage-topics/): `topic.yaml` ```yaml apiVersion: cluster.redpanda.com/v1alpha2 kind: Topic metadata: name: twitch-chat spec: kafkaApiSpec: brokers: - "redpanda-0.redpanda..svc.cluster.local:9093" - "redpanda-1.redpanda..svc.cluster.local:9093" - "redpanda-2.redpanda..svc.cluster.local:9093" tls: caCertSecretRef: name: "redpanda-default-cert" key: "ca.crt" ``` 2. Apply the Topic resource in the same namespace as your Redpanda cluster: ```bash kubectl apply -f topic.yaml --namespace ``` 3. Check the logs of the Redpanda Operator to confirm that the topic was created: ```bash kubectl logs -l app.kubernetes.io/name=operator -c manager --namespace ``` You should see that the Redpanda Operator reconciled the Topic resource. Example output ```json { "level":"info", "ts":"2023-09-25T16:20:09.538Z", "logger":"TopicReconciler.Reconcile", "msg":"Starting reconcile loop", "controller":"topic", "controllerGroup":"cluster.redpanda.com", "controllerKind":"Topic", "Topic": { "name":"twitch-chat", "namespace":"" }, "namespace":"", "name":"twitch-chat", "reconcileID":"c0cf9abc-a553-48b7-9b6e-2de3cdfb4432" } { "level":"info", "ts":"2023-09-25T16:20:09.581Z", "logger":"TopicReconciler.Reconcile", "msg":"reconciliation finished in 43.436125ms, next run in 3s", "controller":"topic", "controllerGroup":"cluster.redpanda.com", "controllerKind":"Topic", "Topic": { "name":"twitch-chat", "namespace":"" }, "namespace":"", "name":"twitch-chat", "reconcileID":"c0cf9abc-a553-48b7-9b6e-2de3cdfb4432", "result": { "Requeue":false, "RequeueAfter":3000000000 } } ``` ### Helm ```bash internal-rpk topic create twitch-chat ``` Example output: TOPIC STATUS twitch-chat OK 3. Describe the topic: ```bash internal-rpk topic describe twitch-chat ``` Expected output: ```none SUMMARY ======= NAME twitch-chat PARTITIONS 1 REPLICAS 1 CONFIGS ======= KEY VALUE SOURCE cleanup.policy delete DYNAMIC_TOPIC_CONFIG compression.type producer DEFAULT_CONFIG message.timestamp.type CreateTime DEFAULT_CONFIG partition_count 1 DYNAMIC_TOPIC_CONFIG redpanda.datapolicy function_name: script_name: DEFAULT_CONFIG redpanda.remote.read false DEFAULT_CONFIG redpanda.remote.write false DEFAULT_CONFIG replication_factor 1 DYNAMIC_TOPIC_CONFIG retention.bytes -1 DEFAULT_CONFIG retention.ms 604800000 DEFAULT_CONFIG segment.bytes 1073741824 DEFAULT_CONFIG ``` 4. Produce a message to the topic: ```bash internal-rpk topic produce twitch-chat ``` 5. Type a message, then press Enter: ```text Pandas are fabulous! ``` Example output: ```text Produced to partition 0 at offset 0 with timestamp 1663282629789. ``` 6. Press Ctrl+C to finish producing messages to the topic. 7. Consume one message from the topic: ```bash internal-rpk topic consume twitch-chat --num 1 ``` Expected output: Your message is displayed along with its metadata: ```json { "topic": "twitch-chat", "value": "Pandas are fabulous!", "timestamp": 1663282629789, "partition": 0, "offset": 0 } ``` ## [](#explore-your-topic-in-redpanda-console)Explore your topic in Redpanda Console Redpanda Console is a developer-friendly web UI for managing and debugging your Redpanda cluster and your applications. In this step, you use port-forwarding to access Redpanda Console on your local network. > 💡 **TIP** > > Because you’re using the Community Edition of Redpanda Console, you should not expose Redpanda Console outside your local network. The Community Edition of Redpanda Console does not provide authentication, and it connects to the Redpanda cluster as superuser. To use the Enterprise Edition, you need a license key. See [Redpanda Licensing](https://docs.redpanda.com/streaming/25.3/get-started/licensing/). 1. Expose Redpanda Console to your localhost: ```bash kubectl --namespace port-forward svc/redpanda-console 8080:8080 ``` The `kubectl port-forward` command actively runs in the command-line window. To execute other commands while the command is running, open another command-line window. 2. Open Redpanda Console on [http://localhost:8080](http://localhost:8080). All your Redpanda brokers are listed along with their IP addresses and IDs. 3. Go to **Topics** > **twitch-chat**. The message that you produced to the topic is displayed along with some other details about the topic. 4. Press Ctrl+C in the command-line to stop the port-forwarding process. ## [](#configure-external-access-to-redpanda)Configure external access to Redpanda If you want to connect to the Redpanda cluster with external clients, Redpanda brokers must advertise an externally accessible address that external clients can connect to. External clients are common in Internet of Things (IoT) environments, or if you use external services that do not implement VPC peering in your network. When you created the cluster, you set the `external.domain` configuration to `customredpandadomain.local`, which means that your Redpanda brokers are advertising the following addresses: - `redpanda-0.customredpandadomain.local` - `redpanda-1.customredpandadomain.local` - `redpanda-2.customredpandadomain.local` To access your Redpanda brokers externally, you can map your worker nodes' IP addresses to these domains. > ⚠️ **CAUTION** > > IP addresses can change. If the IP addresses of your worker nodes change, you must update your `/etc/hosts` file with the new mappings. > > In a production environment, it’s a best practice to use ExternalDNS to manage DNS records for your brokers. See [Use ExternalDNS for external access](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-requirements/#use-externaldns-for-external-access). > 📝 **NOTE** > > These steps work only on Linux operating systems. 1. Add mappings in your `/etc/hosts` file between your worker nodes' IP addresses and their custom domain names: ```bash sudo true && kubectl --namespace get endpoints,node -A -o go-template='{{ range $_ := .items }}{{ if and (eq .kind "Endpoints") (eq .metadata.name "redpanda-external") }}{{ range $_ := (index .subsets 0).addresses }}{{ $nodeName := .nodeName }}{{ $podName := .targetRef.name }}{{ range $node := $.items }}{{ if and (eq .kind "Node") (eq .metadata.name $nodeName) }}{{ range $_ := .status.addresses }}{{ if eq .type "InternalIP" }}{{ .address }} {{ $podName }}.customredpandadomain.local{{ "\n" }}{{ end }}{{ end }}{{ end }}{{ end }}{{ end }}' | envsubst | sudo tee -a /etc/hosts ``` `/etc/hosts` 203.0.113.3 redpanda-0.customredpandadomain.local 203.0.113.5 redpanda-1.customredpandadomain.local 203.0.113.7 redpanda-2.customredpandadomain.local 2. Save the root certificate authority (CA) to your local file system outside Kubernetes: ```bash kubectl --namespace get secret redpanda-external-root-certificate -o go-template='{{ index .data "ca.crt" | base64decode }}' > ca.crt ``` 3. Install `rpk` on your local Linux machine, not on a Pod: ### amd64 ```bash curl -LO https://github.com/redpanda-data/redpanda/releases/latest/download/rpk-linux-amd64.zip && mkdir -p ~/.local/bin && export PATH="~/.local/bin:$PATH" && unzip rpk-linux-amd64.zip -d ~/.local/bin/ ``` ### arm64 ```bash curl -LO https://github.com/redpanda-data/redpanda/releases/latest/download/rpk-linux-arm64.zip && mkdir -p ~/.local/bin && export PATH="~/.local/bin:$PATH" && unzip rpk-linux-arm64.zip -d ~/.local/bin/ ``` > 💡 **TIP** > > You can use `rpk` on Windows only with [WSL](https://learn.microsoft.com/windows/wsl/install). However, commands that require Redpanda to be installed on your machine are not supported, such as [`rpk container`](https://docs.redpanda.com/streaming/current/reference/rpk/rpk-container/rpk-container/) commands, [`rpk iotune`](https://docs.redpanda.com/streaming/current/reference/rpk/rpk-iotune/), and [`rpk redpanda`](https://docs.redpanda.com/streaming/current/reference/rpk/rpk-redpanda/rpk-redpanda/) commands. 4. Configure `rpk` to connect to your cluster using the [pre-configured profile](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/networking/k-connect-to-redpanda/#rpk-profile): ```bash rpk profile create --from-profile <(kubectl get configmap --namespace redpanda-rpk -o go-template='{{ .data.profile }}') ``` Replace `` with the name that you want to give this `rpk` profile. 5. Test the connection: ```bash rpk cluster info ``` ## [](#explore-the-default-kubernetes-components)Explore the default Kubernetes components By default, the Redpanda Helm chart deploys the following Kubernetes components: - [A StatefulSet](#statefulset) with three Pods. - [One PersistentVolumeClaim](#persistentvolumeclaim) for each Pod, each with a capacity of 20Gi. - [A headless ClusterIP Service and a NodePort Service](#service) for each Kubernetes node that runs a Redpanda broker. - [Self-Signed TLS Certificates](#tls-certificates). ### [](#statefulset)StatefulSet Redpanda is a stateful application. Each Redpanda broker needs to store its own state (topic partitions) in its own storage volume. As a result, the Helm chart deploys a StatefulSet to manage the Pods in which the Redpanda brokers are running. ```bash kubectl get statefulset --namespace ``` Example output: NAME READY AGE redpanda 3/3 3m11s StatefulSets ensure that the state associated with a particular Pod replica is always the same, no matter how often the Pod is recreated. Each Pod is also given a unique ordinal number in its name such as `redpanda-0`. A Pod with a particular ordinal number is always associated with a PersistentVolumeClaim with the same number. When a Pod in the StatefulSet is deleted and recreated, it is given the same ordinal number and so it mounts the same storage volume as the deleted Pod that it replaced. ```bash kubectl get pod --namespace ``` Expected output: ```none NAME READY STATUS RESTARTS AGE redpanda-0 1/1 Running 0 6m9s redpanda-1 1/1 Running 0 6m9s redpanda-2 1/1 Running 0 6m9s redpanda-console-5ff45cdb9b-6z2vs 1/1 Running 0 5m redpanda-configuration-smqv7 0/1 Completed 0 6m9s ``` > 📝 **NOTE** > > The `redpanda-configuration` job updates the Redpanda runtime configuration. ### [](#persistentvolumeclaim)PersistentVolumeClaim Redpanda brokers must be able to store their data on disk. By default, the Helm chart uses the default StorageClass in the Kubernetes cluster to create a PersistentVolumeClaim for each Pod. The default StorageClass in your Kubernetes cluster depends on the Kubernetes platform that you are using. ```bash kubectl get persistentvolumeclaims --namespace ``` Expected output: ```none NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE datadir-redpanda-0 Bound pvc-3311ade3-de84-4027-80c6-3d8347302962 20Gi RWO standard 75s datadir-redpanda-1 Bound pvc-4ea8bc03-89a6-41e4-b985-99f074995f08 20Gi RWO standard 75s datadir-redpanda-2 Bound pvc-45c3555f-43bc-48c2-b209-c284c8091c45 20Gi RWO standard 75s ``` ### [](#service)Service The clients writing to or reading from a given partition have to connect directly to the leader broker that hosts the partition. As a result, clients need to be able to connect directly to each Pod. To allow internal and external clients to connect to each Pod that hosts a Redpanda broker, the Helm chart configures two Services: - Internal using the [Headless ClusterIP](#headless-clusterip-service) - External using the [NodePort](#nodeport-service) ```bash kubectl get service --namespace ``` Expected output: ```none NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE redpanda ClusterIP None 5m37s redpanda-console ClusterIP 10.0.251.204 8080 5m redpanda-external NodePort 10.96.137.220 9644:31644/TCP,9094:31092/TCP,8083:30082/TCP,8080:30081/TCP 5m37s ``` #### [](#headless-clusterip-service)Headless ClusterIP Service The headless Service associated with a StatefulSet gives the Pods their network identity in the form of a fully qualified domain name (FQDN). Both Redpanda brokers in the same Redpanda cluster and clients within the same Kubernetes cluster use this FQDN to communicate with each other. An important requirement of distributed applications such as Redpanda is peer discovery: The ability for each broker to find other brokers in the same cluster. When each Pod is rolled out, its `seed_servers` field is updated with the FQDN of each Pod in the cluster so that they can discover each other. ```bash kubectl --namespace exec redpanda-0 -c redpanda -- cat etc/redpanda/redpanda.yaml ``` ```yaml redpanda: data_directory: /var/lib/redpanda/data empty_seed_starts_cluster: false seed_servers: - host: address: redpanda-0.redpanda..svc.cluster.local. port: 33145 - host: address: redpanda-1.redpanda..svc.cluster.local. port: 33145 - host: address: redpanda-2.redpanda..svc.cluster.local. port: 33145 ``` #### [](#nodeport-service)NodePort Service External access is made available by a NodePort service that opens the following ports by default: | Listener | Node Port | Container Port | | --- | --- | --- | | Schema Registry | 30081 | 8081 | | HTTP Proxy | 30082 | 8083 | | Kafka API | 31092 | 9094 | | Admin API | 31644 | 9644 | To learn more, see [Networking and Connectivity in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/networking/k-networking-and-connectivity/). ### [](#tls-certificates)TLS Certificates By default, TLS is enabled in the Redpanda Helm chart. The Helm chart uses [cert-manager](https://cert-manager.io/docs/) to generate four Certificate resources that provide Redpanda with self-signed certificates for internal and external connections. Having separate certificates for internal and external connections provides security isolation. If an external certificate or its corresponding private key is compromised, it doesn’t affect the security of internal communications. ```bash kubectl get certificate --namespace ``` NAME READY redpanda-default-cert True redpanda-default-root-certificate True redpanda-external-cert True redpanda-external-root-certificate True - `redpanda-default-cert`: Self-signed certificate for internal communications. - `redpanda-default-root-certificate`: Root certificate authority for the internal certificate. - `redpanda-external-cert`: Self-signed certificate for external communications. - `redpanda-external-root-certificate`: Root certificate authority for the external certificate. By default, all listeners are configured with the same certificate. To configure separate TLS certificates for different listeners, see [TLS for Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/security/tls/). > 📝 **NOTE** > > The Redpanda Helm chart provides self-signed certificates for convenience. In a production environment, it’s best to use certificates from a trusted Certificate Authority (CA) or integrate with your existing CA infrastructure. ## [](#uninstall-redpanda)Uninstall Redpanda When you finish testing Redpanda, you can uninstall it from your Kubernetes cluster. The steps depend on how you installed Redpanda: using the Redpanda Operator or the Redpanda Helm chart. ### Operator Follow the steps in **exact order** to avoid race conditions between the Redpanda Operator’s reconciliation loop and Kubernetes garbage collection. 1. Delete all Redpanda-related custom resources: ```bash kubectl delete users --namespace --all kubectl delete topics --namespace --all kubectl delete schemas --namespace --all kubectl delete redpanda --namespace --all ``` 2. Make sure requests for those resources return no results. For example, if you had a Redpanda cluster named `redpanda` in the namespace ``, run: ```bash kubectl get redpanda --namespace ``` 3. Uninstall the Redpanda Operator Helm release: ```bash helm uninstall redpanda-controller --namespace ``` Helm does not uninstall CRDs by default when using `helm uninstall` to avoid accidentally deleting existing custom resources. 4. Remove the CRDs. 1. List all Redpanda CRDs installed by the operator: ```bash kubectl api-resources --api-group='cluster.redpanda.com' ``` This command displays all CRDs defined by the Redpanda Operator. For example: ```bash NAME SHORTNAMES APIVERSION NAMESPACED KIND redpandas rp cluster.redpanda.com/v1alpha2 true Redpanda schemas sc cluster.redpanda.com/v1alpha2 true Schema topics cluster.redpanda.com/v1alpha2 true Topic users rpu cluster.redpanda.com/v1alpha2 true User ``` 2. Delete the CRDs: ```bash kubectl get crds -o name | grep cluster.redpanda.com | xargs kubectl delete ``` This command lists all CRDs with the `cluster.redpanda.com` domain suffix and deletes them, ensuring only Redpanda CRDs are removed. Helm does not delete CRDs automatically to prevent data loss, so you must run this step manually. 5. (Optional) Delete any leftover PVCs or Secrets in the namespace: > ⚠️ **CAUTION** > > The following command deletes all PVCs and Secrets in the namespace, which may remove unrelated resources if the namespace is shared with other applications. ```bash kubectl delete pvc,secret --all --namespace ``` ### Helm If you deployed Redpanda with the Redpanda Helm chart, follow these steps to uninstall it: 1. Uninstall the Helm release: ```bash helm uninstall redpanda --namespace ``` 2. (Optional) Delete any leftover PVCs or Secrets in the namespace: > ⚠️ **CAUTION** > > The following command deletes all PVCs and Secrets in the namespace, which may remove unrelated resources if the namespace is shared with other applications. ```bash kubectl delete pvc,secret --all --namespace ``` ## [](#delete-the-cluster)Delete the cluster To delete your Kubernetes cluster: ### kind ```bash kind delete cluster ``` ### minikube ```bash minikube delete ``` To remove the convenience alias created during the quickstart: ```bash unalias internal-rpk ``` ## [](#troubleshoot)Troubleshoot Before troubleshooting your cluster, make sure that you have all the [prerequisites](#prerequisites). ### [](#helm-v3-18-0-is-not-supported-json-number-error)Helm v3.18.0 is not supported (json.Number error) If you are using Helm v3.18.0, you may encounter errors such as: Error: INSTALLATION FAILED: execution error at (redpanda/templates/entry-point.yaml:17:4): invalid Quantity expected string or float64 got: json.Number (1) This is due to a bug in Helm v3.18.0. To avoid similar errors, upgrade to a later version. For more details, see the [Helm GitHub issue](https://github.com/helm/helm/issues/30880). === StatefulSet never rolls out If the StatefulSet Pods remain in a pending state, they are waiting for resources to become available. To identify the Pods that are pending, use the following command: ```bash kubectl get pod --namespace ``` The response includes a list of Pods in the StatefulSet and their status. To view logs for a specific Pod, use the following command. ```bash kubectl logs -f --namespace ``` You can use the output to debug your deployment. ### [](#didnt-match-pod-anti-affinity-rules)Didn’t match pod anti-affinity rules If you see this error, your cluster does not have enough nodes to satisfy the anti-affinity rules: Warning FailedScheduling 18m default-scheduler 0/1 nodes are available: 1 node(s) didn't match pod anti-affinity rules. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod. The Helm chart configures default `podAntiAffinity` rules to make sure that only one Pod running a Redpanda broker is scheduled on each worker node. To learn why, see [Number of workers](https://docs.redpanda.com/streaming/25.3/deploy/redpanda/kubernetes/k-requirements/#number-of-workers). To resolve this issue, do one of the following: - Create additional worker nodes. - Modify the anti-affinity rules (for development purposes only). If adding nodes is not an option, you can modify the `podAntiAffinity` rules in your StatefulSet to be less strict. #### Operator `redpanda-cluster.yaml` ```yaml apiVersion: cluster.redpanda.com/v1alpha2 kind: Redpanda metadata: name: redpanda spec: chartRef: {} clusterSpec: statefulset: podAntiAffinity: type: soft ``` ```bash kubectl apply -f redpanda-cluster.yaml --namespace ``` #### Helm ##### --values `docker-repo.yaml` ```yaml statefulset: podAntiAffinity: type: soft ``` ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --values docker-repo.yaml ``` ##### --set ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --set statefulset.podAntiAffinity.type=soft ``` ### [](#unable-to-mount-volume)Unable to mount volume If you see volume mounting errors in the Pod events or in the Redpanda logs, ensure that each of your Pods has a volume available in which to store data. - If you’re using StorageClasses with dynamic provisioners (default), ensure they exist: ```bash kubectl get storageclass ``` - If you’re using PersistentVolumes, ensure that you have one PersistentVolume available for each Redpanda broker, and that each one has the storage capacity that’s set in `storage.persistentVolume.size`: ```bash kubectl get persistentvolume --namespace ``` To learn how to configure different storage volumes, see [Configure Storage](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/storage/k-configure-storage/). ### [](#failed-to-pull-image)Failed to pull image When deploying the Redpanda Helm chart, you may encounter Docker rate limit issues because the default registry URL is not recognized as a Docker Hub URL. The domain `docker.redpanda.com` is used for statistical purposes, such as tracking the number of downloads. It mirrors Docker Hub’s content while providing specific analytics for Redpanda. Failed to pull image "docker.redpanda.com/redpandadata/redpanda:v": rpc error: code = Unknown desc = failed to pull and unpack image "docker.redpanda.com/redpandadata/redpanda:v": failed to copy: httpReadSeeker: failed open: unexpected status code 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit To fix this error, do one of the following: - Replace the `image.repository` value in the Helm chart with `docker.io/redpandadata/redpanda`. Switching to Docker Hub avoids the rate limit issues associated with `docker.redpanda.com`. #### Operator `redpanda-cluster.yaml` ```yaml apiVersion: cluster.redpanda.com/v1alpha2 kind: Redpanda metadata: name: redpanda spec: chartRef: {} clusterSpec: image: repository: docker.io/redpandadata/redpanda ``` ```bash kubectl apply -f redpanda-cluster.yaml --namespace ``` #### Helm ##### --values `docker-repo.yaml` ```yaml image: repository: docker.io/redpandadata/redpanda ``` ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --values docker-repo.yaml ``` ##### --set ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --set image.repository=docker.io/redpandadata/redpanda ``` - Authenticate to Docker Hub by logging in with your Docker Hub credentials. The `docker.redpanda.com` site acts as a reflector for Docker Hub. As a result, when you log in with your Docker Hub credentials, you will bypass the rate limit issues. ### [](#dig-not-defined)Dig not defined This error means that you are using an unsupported version of [Helm](https://helm.sh/docs/intro/install/): Error: parse error at (redpanda/templates/statefulset.yaml:203): function "dig" not defined To fix this error, ensure that you are using the minimum required version: 3.10.0. ```bash helm version ``` ### [](#repository-name-already-exists)Repository name already exists If you see this error, remove the `redpanda` chart repository, then try installing it again. ```bash helm repo remove redpanda helm repo add redpanda https://charts.redpanda.com helm repo update ``` ### [](#fatal-error-during-checker-data-directory-is-writable-execution)Fatal error during checker "Data directory is writable" execution This error appears when Redpanda does not have write access to your configured storage volume under `storage` in the Helm chart. Error: fatal error during checker "Data directory is writable" execution: open /var/lib/redpanda/data/test\_file: permission denied To fix this error, set `statefulset.initContainers.setDataDirOwnership.enabled` to `true` so that the initContainer can set the correct permissions on the data directories. ### [](#cannot-patch-redpanda-with-kind-statefulset)Cannot patch "redpanda" with kind StatefulSet This error appears when you run `helm upgrade` with the `--values` flag but do not include all your previous overrides. Error: UPGRADE FAILED: cannot patch "redpanda" with kind StatefulSet: StatefulSet.apps "redpanda" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden To fix this error, include all the value overrides from the previous installation using either the `--set` or the `--values` flags. > ⚠️ **WARNING** > > Do not use the `--reuse-values` flag to upgrade from one version of the Helm chart to another. This flag stops Helm from using any new values in the upgraded chart. ### [](#cannot-patch-redpanda-console-with-kind-deployment)Cannot patch "redpanda-console" with kind Deployment This error appears if you try to upgrade your deployment and you already have `console.enabled` set to `true`. Error: UPGRADE FAILED: cannot patch "redpanda-console" with kind Deployment: Deployment.apps "redpanda-console" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map\[string\]string{"app.kubernetes.io/instance":"redpanda", "app.kubernetes.io/name":"console"}, MatchExpressions:\[\]v1.LabelSelectorRequirement(nil)}: field is immutable To fix this error, set `console.enabled` to `false` so that Helm doesn’t try to deploy Redpanda Console again. ### [](#helm-is-in-a-pending-rollback-state)Helm is in a pending-rollback state An interrupted Helm upgrade process can leave your Helm release in a `pending-rollback` state. This state prevents further actions like upgrades, rollbacks, or deletions through standard Helm commands. To fix this: 1. Identify the Helm release that’s in a `pending-rollback` state: ```bash helm list --namespace --all ``` Look for releases with a status of `pending-rollback`. These are the ones that need intervention. 2. Verify the Secret’s status to avoid affecting the wrong resource: ```bash kubectl --namespace get secret --show-labels ``` Identify the Secret associated with your Helm release by its `pending-rollback` status in the labels. > ⚠️ **WARNING** > > Ensure you have correctly identified the Secret to avoid unintended consequences. Deleting the wrong Secret could impact other deployments or services. 3. Delete the Secret to clear the `pending-rollback` state: ```bash kubectl --namespace delete secret -l status=pending-rollback ``` After clearing the `pending-rollback` state: - **Retry the upgrade**: Restart the upgrade process. You should investigate the initial failure to avoid getting into the `pending-rollback` state again. - **Perform a rollback**: If you need to roll back to a previous release, use `helm rollback ` to revert to a specific, stable release version. ### [](#crash-loop-backoffs)Crash loop backoffs If a broker crashes after startup, or gets stuck in a crash loop, it can accumulate an increasing amount of stored state. This accumulated state not only consumes additional disk space but also prolongs the time required for each subsequent restart to process it. To prevent infinite crash loops, the Redpanda Helm chart sets the [`crash_loop_limit`](https://docs.redpanda.com/streaming/25.3/reference/properties/broker-properties/#crash_loop_limit) broker configuration property to `5`. The crash loop limit is the number of consecutive crashes that can happen within one hour of each other. By default, the broker terminates immediately after hitting the `crash_loop_limit`. The Pod running Redpanda remains in a `CrashLoopBackoff` state until its internal consecutive crash counter is reset to zero. To facilitate debugging in environments where a broker is stuck in a crash loop, you can also set the [`crash_loop_sleep_sec`](https://docs.redpanda.com/streaming/25.3/reference/properties/broker-properties/#crash_loop_sleep_sec) broker configuration property. This setting determines how long the broker sleeps before terminating the process after reaching the crash loop limit. By providing a window during which the Pod remains available, you can SSH into it and troubleshoot the issue. Example configuration: ```yaml config: node: crash_loop_limit: 5 crash_loop_sleep_sec: 60 ``` In this example, when the broker hits the `crash_loop_limit` of 5, it will sleep for 60 seconds before terminating the process. This delay allows administrators to access the Pod and troubleshoot. To troubleshoot a crash loop backoff: 1. Check the Redpanda logs from the most recent crashes: ```bash kubectl logs --namespace ``` > 📝 **NOTE** > > Kubernetes retains logs only for the current and the previous instance of a container. This limitation makes it difficult to access logs from earlier crashes, which may contain vital clues about the root cause of the issue. Given these log retention limitations, setting up a centralized logging system is crucial. Systems such as [Loki](https://grafana.com/docs/loki/latest/) or [Datadog](https://www.datadoghq.com/product/log-management/) can capture and store logs from all containers, ensuring you have access to historical data. 2. Resolve the issue that led to the crash loop backoff. 3. Reset the crash counter to zero to allow Redpanda to restart. You can do any of the following to reset the counter: - Make changes to any of the following sections in the Redpanda Helm chart to trigger an update: - `config.node` - `config.tunable` For example: ```yaml config: node: crash_loop_limit: ``` - Delete the `startup_log` file in the broker’s data directory. ```bash kubectl exec --namespace -- rm /var/lib/redpanda/data/startup_log ``` > 📝 **NOTE** > > It might be challenging to execute this command within a Pod that is in a `CrashLoopBackoff` state due to the limited time during which the Pod is available before it restarts. Wrapping the command in a loop might work. - Wait one hour since the last crash. The crash counter resets after one hour. To avoid future crash loop backoffs and manage the accumulation of small segments effectively: - [Monitor](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/monitoring/k-monitor-redpanda/) the size and number of segments regularly. - Optimize your Redpanda configuration for segment management. - Consider implementing [Tiered Storage](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/tiered-storage/k-tiered-storage/) to manage data more efficiently. ### [](#a-redpanda-enterprise-edition-license-is-required)A Redpanda Enterprise Edition license is required During a Redpanda upgrade, if enterprise features are enabled and a valid Enterprise Edition license is missing, Redpanda logs a warning and aborts the upgrade process on the first broker. This issue prevents a successful upgrade. A Redpanda Enterprise Edition license is required to use the currently enabled features. To apply your license, downgrade this broker to the pre-upgrade version and provide a valid license key via rpk using 'rpk cluster license set ', or via Redpanda Console. To request an enterprise license, please visit . To try Redpanda Enterprise for 30 days, visit . For more information, see . If you encounter this message, follow these steps to recover: 1. [Roll back the affected broker to the original version](https://docs.redpanda.com/streaming/25.3/upgrade/k-rolling-upgrade/#roll-back). 2. Do one of the following: - [Apply a valid Redpanda Enterprise Edition license](https://docs.redpanda.com/streaming/25.3/get-started/licensing/add-license-redpanda/) to the cluster. - Disable enterprise features. If you do not have a valid license and want to proceed without using enterprise features, you can disable the enterprise features in your Redpanda configuration. 3. Retry the upgrade. For more troubleshooting steps, see [Troubleshoot Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/troubleshoot/errors-solutions/k-resolve-errors/). ## [](#next-steps)Next steps - [Try an example in Redpanda Labs](https://docs.redpanda.com/labs/) - [Learn more about Redpanda Console](https://docs.redpanda.com/streaming/25.3/manage/console/) - [Learn more about rpk](https://docs.redpanda.com/streaming/25.3/get-started/rpk-install/) > 💡 **TIP** > > When you’re ready to use a registered domain, make sure to remove your entries from the `/etc/hosts` file, and see [Configure External Access through a NodePort Service](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/networking/external/k-nodeport/#use-the-default-redpanda-subdomains). ## [](#suggested-reading)Suggested reading - [Networking and Connectivity in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/networking/k-networking-and-connectivity/) - [Configure TLS for Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/security/tls/) - [Configure SASL for Redpanda in Kubernetes](https://docs.redpanda.com/streaming/25.3/manage/kubernetes/security/authentication/k-authentication/) - [Redpanda Helm Specification](https://docs.redpanda.com/streaming/25.3/reference/k-redpanda-helm-spec/) - [Redpanda CRD Reference](https://docs.redpanda.com/streaming/25.3/reference/k-crd/) - [Redpanda Console README](https://github.com/redpanda-data/console) on GitHub ## Suggested labs - [Disaster Recovery with Envoy and Shadowing](https://docs.redpanda.com/labs/docker-compose/envoy-shadowing/) - [Redpanda Iceberg Docker Compose Example](https://docs.redpanda.com/labs/docker-compose/iceberg/) - [Stream Jira Issues to Redpanda for Real-Time Metrics](https://docs.redpanda.com/labs/docker-compose/jira-metrics-pipeline/) - [Migrate Data with Redpanda Migrator](https://docs.redpanda.com/labs/docker-compose/redpanda-migrator/) - [Start a Single Redpanda Broker with Redpanda Console in Docker](https://docs.redpanda.com/labs/docker-compose/single-broker/) - [Start a Cluster of Redpanda Brokers with Redpanda Console in Docker](https://docs.redpanda.com/labs/docker-compose/three-brokers/) - [Set Up GitOps for the Redpanda Helm Chart](https://docs.redpanda.com/labs/kubernetes/gitops-helm/) - [Iceberg Streaming on Kubernetes with Redpanda, MinIO, and Spark](https://docs.redpanda.com/labs/kubernetes/iceberg/) - [Set Up MySQL CDC with Debezium and Redpanda](https://docs.redpanda.com/labs/docker-compose/cdc-mysql-json/) - [Set Up Postgres CDC with Debezium and Redpanda](https://docs.redpanda.com/labs/docker-compose/cdc-postgres-json/) See more [Search all labs](https://docs.redpanda.com/labs)