Docs Self-Managed Deploy Self-Hosted Kubernetes Deploy Redpanda This is documentation for Self-Managed v23.2, which is no longer supported. To view the latest available version of the docs, see v24.2. Deploy Redpanda in Kubernetes This topic describes how to configure and deploy one or more Redpanda clusters in Kubernetes. Prerequisites Make sure that your Kubernetes cluster meets the requirements. Deploy a Redpanda cluster To deploy Redpanda, you can use the following methods: Helm and the Redpanda Operator: Use the Redpanda Operator to manage your Redpanda cluster. To deploy a Redpanda cluster, you apply a Redpanda resource, which is used by the underlying Flux controllers to deploy the Redpanda Helm chart. Helm: Deploy the Redpanda Helm chart directly. Regardless of the method you choose to deploy Redpanda, you’ll deploy the Redpanda Helm chart, which includes Redpanda and Redpanda Console. Redpanda Console comes bundled as a subchart within the Redpanda Helm chart. For more details about the differences between these two methods, see Redpanda in Kubernetes. Helm + Operator Helm The Redpanda Operator handles the deployment and management of the Redpanda Helm chart for you using Flux. The Redpanda Operator extends Kubernetes with custom resource definitions (CRDs), allowing you to define Redpanda clusters as native Kubernetes resources. The resource that the Redpanda Operator uses to represent a Redpanda cluster is the Redpanda resource. When you deploy a Redpanda resource, the Redpanda Operator takes that configuration and passes it to Flux. Flux, in turn, interacts with Helm, creating the necessary HelmRepository and HelmRelease resources to deploy and manage the Redpanda Helm chart. The Redpanda Operator is namespace scoped. You must install the Redpanda Operator in the same namespace as your Redpanda resource (Redpanda cluster). Make sure that you have permission to install custom resource definitions (CRDs): kubectl auth can-i create CustomResourceDefinition --all-namespaces You should see yes in the output. You need these cluster-level permissions to install cert-manager and Redpanda Operator CRDs in the next steps. Install cert-manager: helm repo add jetstack https://charts.jetstack.io helm repo update helm install cert-manager jetstack/cert-manager \ --set installCRDs=true \ --namespace cert-manager \ --create-namespace The Redpanda Helm chart enables TLS by default and uses cert-manager to manage TLS certificates. Install the Redpanda Operator CRDs: kubectl kustomize "https://github.com/redpanda-data/redpanda-operator//src/go/k8s/config/crd?ref=v2.1.10-23.2.18" \ | kubectl apply -f - Deploy the Redpanda Operator. helm repo add redpanda https://charts.redpanda.com helm upgrade --install redpanda-controller redpanda/operator \ --namespace <namespace> \ --set image.tag=v2.1.10-23.2.18 \ --create-namespace Ensure that the Deployment is successfully rolled out: kubectl --namespace <namespace> rollout status --watch deployment/redpanda-controller-operator deployment "redpanda-controller-operator" successfully rolled out Install a Redpanda custom resource to deploy a Redpanda cluster and Redpanda Console. redpanda-cluster.yaml apiVersion: cluster.redpanda.com/v1alpha1 kind: Redpanda metadata: name: redpanda spec: chartRef: {} clusterSpec: {} metadata.name: Name to assign the Redpanda cluster. This name is also assigned to the Helm release. spec.chartRef: Information about the Helm chart that will be used to deploy Redpanda. spec.clusterSpec: This is where you can override default values in the Redpanda Helm chart. See Configuration advice for details. Apply the Redpanda resource: kubectl apply -f redpanda-cluster.yaml --namespace <namespace> The Redpanda resource must be deployed in the same namespace as the Redpanda Operator. Each new deployment of Redpanda requires a separate namespace. Wait for the Redpanda Operator to deploy Redpanda using the Helm chart: kubectl get redpanda --namespace <namespace> --watch NAME READY STATUS redpanda True Redpanda reconciliation succeeded This step may take a few minutes. You can watch for new Pods to make sure that the deployment is progressing: kubectl get pod --namespace <namespace> If it’s taking too long, see Troubleshooting. Verify that each Redpanda broker is scheduled on only one Kubernetes node: kubectl get pod --namespace <namespace> \ -o=custom-columns=NODE:.spec.nodeName,NAME:.metadata.name -l \ app.kubernetes.io/component=redpanda-statefulset Expected output: example-worker3 redpanda-0 example-worker2 redpanda-1 example-worker redpanda-2 Helm is a package manager for Kubernetes, which simplifies the process of defining, installing, and upgrading Kubernetes applications. Helm uses charts, a collection of files that describe a related set of Kubernetes resources, to deploy applications in a Kubernetes cluster. Install cert-manager using Helm: helm repo add jetstack https://charts.jetstack.io helm repo update helm install cert-manager jetstack/cert-manager \ --set installCRDs=true \ --namespace cert-manager \ --create-namespace The Redpanda Helm chart enables TLS by default and uses cert-manager to manage TLS certificates. Install the Redpanda Helm chart to deploy a Redpanda cluster and Redpanda Console. Configure any additional Helm values that you want to override. See Configuration advice for details. helm repo add redpanda https://charts.redpanda.com helm install redpanda redpanda/redpanda \ --namespace <namespace> \ --create-namespace Each deployment of the Redpanda Helm chart requires a separate namespace. Ensure you choose a unique namespace for each deployment. Wait for the Redpanda cluster to be ready: kubectl --namespace <namespace> rollout status statefulset redpanda --watch When the Redpanda cluster is ready, the output should look similar to the following: statefulset rolling update complete 3 pods at revision redpanda-8654f645b4... Verify that each Redpanda broker is scheduled on only one Kubernetes node: kubectl get pod --namespace <namespace> \ -o=custom-columns=NODE:.spec.nodeName,NAME:.metadata.name -l \ app.kubernetes.io/component=redpanda-statefulset Expected output: example-worker3 redpanda-0 example-worker2 redpanda-1 example-worker redpanda-2 Deploy multiple Redpanda clusters You can deploy more than one Redpanda cluster in the same Kubernetes cluster by using a different namespace and unique node ports. Helm + Operator Helm Install another instance of the Redpanda Operator in a different namespace to your existing ones. This Redpanda Operator will manage Redpanda clusters only in its namespace. helm repo add redpanda https://charts.redpanda.com helm upgrade --install redpanda-controller redpanda/operator \ --namespace <another-namespace> \ --set image.tag=v2.1.10-23.2.18 \ --create-namespace Apply a Redpanda resource in the same namespace as your new Redpanda Operator to deploy your new Redpanda cluster. Make sure to use unique node ports for the listeners in your Redpanda resource so that they don’t conflict with any existing node ports in your other Redpanda clusters. See External access. redpanda-cluster-two.yaml apiVersion: cluster.redpanda.com/v1alpha1 kind: Redpanda metadata: name: redpanda-two spec: chartRef: {} clusterSpec: listeners: kafka: external: default: advertisedPorts: [31093] admin: external: default: advertisedPorts: [31645] http: external: default: advertisedPorts: [30083] rpc: port: 33146 schemaRegistry: external: default: advertisedPorts: [30084] Install the Redpanda Helm chart in a different namespace to your existing Redpanda clusters. Make sure to use unique node ports for the listeners in your Redpanda resource so that they don’t conflict with any existing node ports in your other Redpanda clusters. See External access. helm repo add redpanda https://charts.redpanda.com helm install redpanda-two redpanda/redpanda \ --namespace <anothernamespace> \ --set listeners.kafka.external.default.advertisedPorts[0]=31093 \ --set listeners.admin.external.default.advertisedPorts[0]=31645 \ --set listeners.http.external.default.advertisedPorts[0]=30083 \ --set listeners.rpc.port=33146 \ --set listeners.schemaRegistry.external.default.advertisedPorts[0]=30084 --create-namespace Configuration advice This section provides advice for configuring the Redpanda Helm chart. For all available settings, see Redpanda Helm Chart Specification. To learn how to customize the Redpanda Helm chart, see Customize the Helm Chart. Name overrides Deploying multiple instances of the same Helm chart in a Kubernetes cluster can lead to naming conflicts. Using nameOverride and fullnameOverride helps differentiate between them. If you have a production and staging environment for Redpanda, different names help to avoid confusion. Use nameOverride to customize the labels app.kubernetes.io/component=<nameOverride>-statefulset and app.kubernetes.io/name=<nameOverride>. Use fullnameOverride to customize the name of the StatefulSet and Services. nameOverride: 'redpanda-production' fullnameOverride: 'redpanda-instance-prod' For all available settings, see the Helm specification. Labels Kubernetes labels help you to organize, query, and manage your resources. Use labels to categorize Kubernetes resources in different deployments by environment, purpose, or team. commonLabels: env: 'production' For all available settings, see the Helm specification. Tolerations Tolerations and taints allow Pods to be scheduled onto nodes where they otherwise wouldn’t. If you have nodes dedicated to Redpanda with a taint dedicated=redpanda:NoSchedule, the following toleration allows the Redpanda brokers to be scheduled on them. tolerations: - key: "dedicated" operator: "Equal" value: "redpanda" effect: "NoSchedule" For all available settings, see the Helm specification. Docker image You can specify the image tag to deploy a known version of the Redpanda Docker image. By default, the image tag is set in Chart.appVersion. Avoid using the latest tag, which can lead to unexpected changes. If you’re using a private repository, always ensure your nodes have the necessary credentials to pull the image. image: repository: docker.redpanda.com/redpandadata/redpanda tag: "v23.2.21" imagePullSecrets: [] For all available settings, see the Helm specification. Number of Redpanda brokers By default, the Redpanda Helm chart deploys a StatefulSet with three Pod replicas (Redpanda brokers). In a production cluster, deploy at least three Redpanda brokers to use as seed servers. Seed servers are used to bootstrap the gossip process for new brokers joining a cluster. When a new broker joins, it connects to the seed servers to find out the topology of the Redpanda cluster. A larger number of seed servers makes consensus more robust and minimizes the chance of unwanted clusters forming when brokers are restarted without any data. statefulset: replicas: 3 You must provision one dedicated worker node for each Redpanda broker that you plan to deploy in your Redpanda cluster. The Redpanda Helm chart configures podAntiAffinity rules to make sure that each Redpanda broker runs on its own worker node. For all available settings, see the Helm specification. See also: High Availability in Kubernetes Kubernetes Cluster Requirements TLS By default, the Helm chart enables TLS (Transport Layer Security) for encrypted communication. Internal (default) and external (external) self-signed certificates are generated using cert-manager. See TLS Certificates. tls: enabled: true certs: # This key represents the name of the certificate. default: caEnabled: true # This key represents the name of the certificate. external: caEnabled: true For all available settings, see the Helm specification. See also: TLS for Redpanda in Kubernetes Authentication If you want to authenticate clients connections to the Redpanda cluster, you can enable SASL authentication. auth: sasl: enabled: true mechanism: "SCRAM-SHA-512" secretRef: "sasl-password-secret" users: [] For all available settings, see the Helm specification. See also: Configure Authentication for Redpanda in Kubernetes Resources By default, the resources allocated to Redpanda are for a development environment. In a production cluster, the resources you allocate should be proportionate to your machine type. You should determine and set these values before deploying the cluster. resources: cpu: cores: 4 memory: enable_memory_locking: true container: max: 10Gi For all available settings, see the Helm specification. See also: Manage Pod Resources in Kubernetes Kubernetes Cluster Requirements and Recommendations Storage By default, the Redpanda Helm chart creates PersistentVolumeClaims (PVCs) for each Redpanda broker in the StatefulSet. The PVC uses the default StorageClass in your cluster. In production, it’s best to use local PersistentVolumes (PVs) that are backed by NVMe devices to store the Redpanda data directory. NVMe devices outperform traditional SSDs or HDDs. For cloud instance types that support NVMe disks, see Kubernetes Cluster Requirements and Recommendations. Redpanda Data recommends creating StorageClasses that use the local volume manager (LVM) CSI driver to automatically provision PVs. The LVM allows you to group physical storage devices into a logical volume group. Allocating logical volumes from a logical volume group provides greater flexibility in terms of storage expansion and management. The LVM supports features such as resizing, snapshots, and striping, which are not available with the other drivers such as the local volume static provisioner. storage: persistentVolume: enabled: true size: 100Gi storageClass: csi-driver-lvm-striped-xfs For an example of configuring local PersistentVolumes backed by NVMe disks, see one of the following guides: Azure Kubernetes Service (AKS) Elastic Kubernetes Service (EKS) Google Kubernetes Engine (GKE) For all available settings, see the Helm specification. See also: Supported Volume Types for Data in Redpanda Configure Storage for the Redpanda data directory in Kubernetes External access By default, the Redpanda Helm chart deploys a NodePort Service for external access to the Redpanda listeners. You can configure individual node ports for each listener. The NodePort Service provides the lowest latency of all the Kubernetes Services because it does not include any unnecessary routing or middleware. Client connections go to the Redpanda brokers in the most direct way possible, through the worker nodes. By default, the fully qualified domain names (FQDNs) that brokers advertise are their internal addresses within the Kubernetes cluster, which are not reachable from outside the cluster. To make the cluster accessible from outside, each broker must advertise a domain that can be reached from outside the cluster. external: enabled: true type: NodePort For all available settings, see the Helm specification. See also: About Networking and Connectivity in Kubernetes Configure Listeners in Kubernetes ExternalDNS You should use ExternalDNS to manage DNS records for your Pods' domains. ExternalDNS synchronizes exposed Kubernetes Services with various DNS providers, rendering Kubernetes resources accessible through DNS servers. Benefits of ExternalDNS include: Automation: ExternalDNS automatically configures public DNS records when you create, update, or delete Kubernetes Services or Ingresses. This eliminates the need for manual DNS configuration, which can be error-prone. Compatibility: ExternalDNS is compatible with a wide range of DNS providers, including major cloud providers such as AWS, Google Cloud, and Azure, and DNS servers like CoreDNS and PowerDNS. Integration with other tools: ExternalDNS can be used with other Kubernetes tools, such as ingress controllers or cert-manager for managing TLS certificates. external: enabled: true type: LoadBalancer externalDns: enabled: true For all available settings, see the Helm specification. See also: ExternalDNS with a NodePort Service ExternalDNS with LoadBalancer Services Logging By default, the log-level is set to info. In production, use the info logging level to avoid overwhelming the storage. For debugging purposes, temporarily change the logging level to debug. logging: level: "info" For all available settings, see the Helm specification. Monitoring By default, monitoring is disabled. If you have the Prometheus Operator, enable monitoring to deploy a ServiceMonitor resource for Redpanda. Observability is essential in production environments. monitoring: enabled: true For all available settings, see the Helm specification. See also: Monitor in Kubernetes StatefulSet update strategy For smooth and uninterrupted updates, use the default RollingUpdate strategy. Additionally, set a budget to ensure a certain number of replicas stay available during the update. statefulset: updateStrategy: type: "RollingUpdate" budget: maxUnavailable: 1 For all available settings, see the Helm specification. Affinity rules By default, the Redpanda Helm chart also uses podAntiAffinity rules to stop the Kubernetes scheduler from placing multiple Redpanda brokers on the same node. These rules offer two benefits: To minimize the risk of data loss by ensuring that a node’s failure results in the loss of only one Redpanda broker. To prevent resource contention between brokers by ensuring they are never co-located on the same node. Affinities control Pod placement in the cluster based on various conditions. Set these according to your high availability and infrastructure needs. For example, this is a soft rule that tries to ensure the Kubernetes scheduler doesn’t place two Pods with the same app: redpanda label in the same zone. However, if it’s not possible, the scheduler can still place the Pods in the same zone. statefulset: podAntiAffinity: topologyKey: kubernetes.io/hostname type: hard weight: 100 custom: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 podAffinityTerm: labelSelector: matchExpressions: - key: "app" operator: "In" values: - "redpanda" topologyKey: "kubernetes.io/zone" For all available settings, see the Helm specification. See also: High Availability in Kubernetes Graceful shutdown By default, Pods are given 90 seconds to shut down gracefully. If your brokers require additional time for a graceful shutdown, modify the terminationGracePeriodSeconds. statefulset: terminationGracePeriodSeconds: 100 For all available settings, see the Helm specification. See also: Upgrade Redpanda in Kubernetes Service account Restricting permissions is a best practice. Create a dedicated ServiceAccount for each Pod. To assign roles to this ServiceAccount, see Role-based access control (RBAC). serviceAccount: create: true name: "redpanda-service-account" For all available settings, see the Helm specification. Role-based access control (RBAC) RBAC is a method for providing permissions to ServiceAccounts based on roles. Some features such as rack awareness require both a ServiceAccount and RBAC to access resources using the Kubernetes API. rbac: enabled: true annotations: {} If you use the Redpanda Operator, you must also deploy the Redpanda Operator Helm chart with rbac.createRPKBundleCRs set to true to give it the required roles. For all available settings, see the Helm specification. See also: Enable Rack Awareness in Kubernetes Find the latest versions of the Redpanda Helm charts To list the latest version of the Redpanda Helm chart, use the helm search command: helm search repo redpanda To find the versions that are installed on your machine, run the following: helm list --namespace <namespace> Explore the default Kubernetes components By default, the Redpanda Helm chart deploys the following Kubernetes components: A StatefulSet with three Pods. One PersistentVolumeClaim for each Pod, each with a capacity of 20Gi. A headless ClusterIP Service and a NodePort Service for each Kubernetes node that runs a Redpanda broker. Self-Signed TLS Certificates. StatefulSet Redpanda is a stateful application. Each Redpanda broker needs to store its own state (topic partitions) in its own storage volume. As a result, the Helm chart deploys a StatefulSet to manage the Pods in which the Redpanda brokers are running. kubectl get statefulset --namespace <namespace> Example output: NAME READY AGE redpanda 3/3 3m11s StatefulSets ensure that the state associated with a particular Pod replica is always the same, no matter how often the Pod is recreated. Each Pod is also given a unique ordinal number in its name such as redpanda-0. A Pod with a particular ordinal number is always associated with a PersistentVolumeClaim with the same number. When a Pod in the StatefulSet is deleted and recreated, it is given the same ordinal number and so it mounts the same storage volume as the deleted Pod that it replaced. kubectl get pod --namespace <namespace> Expected output: NAME READY STATUS RESTARTS AGE redpanda-0 1/1 Running 0 6m9s redpanda-1 1/1 Running 0 6m9s redpanda-2 1/1 Running 0 6m9s redpanda-console-5ff45cdb9b-6z2vs 1/1 Running 0 5m redpanda-configuration-smqv7 0/1 Completed 0 6m9s The redpanda-configuration job updates the Redpanda runtime configuration. PersistentVolumeClaim Redpanda brokers must be able to store their data on disk. By default, the Helm chart uses the default StorageClass in the Kubernetes cluster to create a PersistentVolumeClaim for each Pod. The default StorageClass in your Kubernetes cluster depends on the Kubernetes platform that you are using. kubectl get persistentvolumeclaims --namespace <namespace> Expected output: NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE datadir-redpanda-0 Bound pvc-3311ade3-de84-4027-80c6-3d8347302962 20Gi RWO standard 75s datadir-redpanda-1 Bound pvc-4ea8bc03-89a6-41e4-b985-99f074995f08 20Gi RWO standard 75s datadir-redpanda-2 Bound pvc-45c3555f-43bc-48c2-b209-c284c8091c45 20Gi RWO standard 75s Service The clients writing to or reading from a given partition have to connect directly to the leader broker that hosts the partition. As a result, clients need to be able to connect directly to each Pod. To allow internal and external clients to connect to each Pod that hosts a Redpanda broker, the Helm chart configures two Services: Internal using the Headless ClusterIP External using the NodePort kubectl get service --namespace <namespace> Expected output: NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE redpanda ClusterIP None <none> <none> 5m37s redpanda-console ClusterIP 10.0.251.204 <none> 8080 5m redpanda-external NodePort 10.96.137.220 <none> 9644:31644/TCP,9094:31092/TCP,8083:30082/TCP,8080:30081/TCP 5m37s Headless ClusterIP Service The headless Service associated with a StatefulSet gives the Pods their network identity in the form of a fully qualified domain name (FQDN). Both Redpanda brokers in the same Redpanda cluster and clients within the same Kubernetes cluster use this FQDN to communicate with each other. An important requirement of distributed applications such as Redpanda is peer discovery: The ability for each broker to find other brokers in the same cluster. When each Pod is rolled out, its seed_servers field is updated with the FQDN of each Pod in the cluster so that they can discover each other. kubectl --namespace <namespace> exec redpanda-0 -c redpanda -- cat etc/redpanda/redpanda.yaml redpanda: data_directory: /var/lib/redpanda/data empty_seed_starts_cluster: false seed_servers: - host: address: redpanda-0.redpanda.<namespace>.svc.cluster.local. port: 33145 - host: address: redpanda-1.redpanda.<namespace>.svc.cluster.local. port: 33145 - host: address: redpanda-2.redpanda.<namespace>.svc.cluster.local. port: 33145 NodePort Service External access is made available by a NodePort service that opens the following ports by default: Listener Node Port Container Port Schema Registry 30081 8081 HTTP Proxy 30082 8083 Kafka API 31092 9094 Admin API 31644 9644 To learn more, see Networking and Connectivity in Kubernetes. TLS Certificates By default, TLS is enabled in the Redpanda Helm chart. The Helm chart uses cert-manager to generate four Certificate resources that provide Redpanda with self-signed certificates for internal and external connections. Having separate certificates for internal and external connections provides security isolation. If an external certificate or its corresponding private key is compromised, it doesn’t affect the security of internal communications. kubectl get certificate --namespace <namespace> NAME READY redpanda-default-cert True redpanda-default-root-certificate True redpanda-external-cert True redpanda-external-root-certificate True redpanda-default-cert: Self-signed certificate for internal communications. redpanda-default-root-certificate: Root certificate authority for the internal certificate. redpanda-external-cert: Self-signed certificate for external communications. redpanda-external-root-certificate: Root certificate authority for the external certificate. By default, all listeners are configured with the same certificate. To configure separate TLS certificates for different listeners, see TLS for Redpanda in Kubernetes. The Redpanda Helm chart provides self-signed certificates for convenience. In a production environment, it’s best to use certificates from a trusted Certificate Authority (CA) or integrate with your existing CA infrastructure. Next steps See the Manage Kubernetes topics to learn how to customize the chart to meet your needs. Suggested reading High Availability in Kubernetes Redpanda Helm Specification Redpanda CRD Reference Back to top × Simple online edits For simple changes, such as fixing a typo, you can edit the content directly on GitHub. Edit on GitHub Or, open an issue to let us know about something that you want us to change. Open an issue Contribution guide For extensive content updates, or if you prefer to work locally, read our contribution guide . Was this helpful? thumb_up thumb_down group Ask in the community mail Share your feedback group_add Make a contribution Tune Worker Nodes Deploy Connectors