# Manage Pod Resources in Kubernetes > For the complete documentation index, see [llms.txt](https://docs.redpanda.com/llms.txt). Component-specific: [streaming-full.txt](https://docs.redpanda.com/streaming-full.txt) --- title: Manage Pod Resources in Kubernetes latest-operator-version: v26.1.4 # EOL = End-of-Life (support lifecycle status) page-is-nearing-eol: "false" page-is-past-eol: "true" page-eol-date: December 22, 2024 latest-console-tag: v3.7.3 latest-connect-version: 4.93.0 docname: kubernetes/k-manage-resources page-component-name: streaming page-version: "23.3" page-component-version: "23.3" page-component-title: Streaming page-relative-src-path: kubernetes/k-manage-resources.adoc page-edit-url: https://github.com/redpanda-data/docs/edit/v/23.3/modules/manage/pages/kubernetes/k-manage-resources.adoc description: Configure your Pod resources such as memory, CPU, and storage. page-git-created-date: "2024-01-04" page-git-modified-date: "2024-04-30" support-status: past end-of-life --- You can define requirements for Pod resources such as CPU, memory, and storage. Redpanda Data recommends that you determine and set these values before deploying the cluster, but you can also update the values on a running cluster. ## [](#prerequisites)Prerequisites - See [Kubernetes Cluster Requirements and Recommendations](https://docs.redpanda.com/streaming/23.3/deploy/deployment-option/self-hosted/kubernetes/k-requirements/) for the minimum worker node, memory, CPU, and storage requirements. - Make sure that your physical or virtual machines have enough resources to give to Redpanda. To see the available resources on the worker nodes that you have provisioned: ```bash kubectl describe nodes ``` ## [](#production-considerations)Production considerations - [Enable CPU pinning](#cpu). This configuration allows Redpanda to maintain exclusive access to designated CPU cores, improving throughput and reducing latency caused by CPU context switches. - [Enable the `Guaranteed` quality of service class for Pods that run Redpanda](#qos). This setup ensures that the CPU and memory allocated to Redpanda are not subject to throttling or other contention issues, providing a stable and predictable performance environment. - [Enable memory locking](#memory). This configuration prevents the operating system from paging out Redpanda’s memory to disk, which can significantly impact performance. ## [](#limitations)Limitations Redpanda does not support decreasing the CPU cores for brokers in an existing cluster. ## [](#cpu)CPU pinning Redpanda is designed to maximize throughput by [pinning its threads to specific CPU cores](https://docs.redpanda.com/streaming/23.3/get-started/architecture/#tpc) (CPU pinning). However, CPU pinning is unavailable by default in Kubernetes because it uses the [Completely Fair Scheduler (CFS)](https://www.kernel.org/doc/html/latest/scheduler/sched-design-CFS.html) to dynamically manage CPU allocation. The CFS distributes CPU time among all processes based on their priority and resource requests, which can lead to frequent context switches and migration of processes across different cores. This behavior can degrade the performance of Redpanda due to: - **Loss of CPU cache affinity**: As processes move across cores, they lose the cache locality, leading to increased cache misses and higher latency. - **Scheduling latency**: Dynamic CPU allocation can introduce delays as the scheduler determines which core a process should run on, based on current load and availability. ![Comparison of context switching between CFS and the static CPU Manager Policy](https://docs.redpanda.com/streaming/23.3/shared/_images/cpu-pinning.png) To enable CPU pinning in Kubernetes, you must enable the [`static` policy](https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/#static-policy) for the CPU manager. With this policy enabled, the CPU Manager controls a shared pool of CPUs. Initially, this shared pool contains all the CPUs in the worker node. However, when a container is created with an integral CPU request in a [`Guaranteed` Pod](https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/#create-a-pod-that-gets-assigned-a-qos-class-of-guaranteed), CPUs for that container are removed from the shared pool and assigned exclusively to that container for its lifetime. Other containers are migrated off these exclusively allocated CPUs. ![High-level view of Guaranteed Pods with exclusive CPUs](https://docs.redpanda.com/streaming/23.3/shared/_images/guaranteed-pod.png) ### [](#enable-cpu-pinning)Enable CPU pinning To enable CPU pinning for Redpanda in Kubernetes: 1. Ensure your Kubernetes cluster is using the [`static` CPU Manager](https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/#static-policy). With this policy enabled, a request for an integral CPU amount in a `Guaranteed` Pod will result in that container being allocated exclusive access to that number of CPUs through [cgroup’s cpuset](https://www.kernel.org/doc/Documentation/cgroup-v1/cpusets.txt). 2. Configure the Redpanda container to request an integral CPU amount: #### Helm + Operator `redpanda-cluster.yaml` ```yaml apiVersion: cluster.redpanda.com/v1alpha1 kind: Redpanda metadata: name: redpanda spec: chartRef: {} clusterSpec: resources: cpu: cores: (1) overprovisioned: false (2) ``` ```bash kubectl apply -f redpanda-cluster.yaml --namespace ``` #### Helm ##### --values `cpu-cores.yaml` ```yaml resources: cpu: cores: (1) overprovisioned: false (2) ``` ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --values cpu-cores.yaml --reuse-values ``` ##### --set ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --set resources.cpu.cores= \ (1) --set resources.cpu.overprovisioned=false (2) ``` > 📝 **NOTE** > > If you use fractional values, [CPU pinning is disabled](#pinning). 3. [Enable the `Guaranteed` QoS class for your Pods that run Redpanda](#qos). Only containers that are both part of a `Guaranteed` Pod and have integer CPU requests are assigned exclusive CPUs. ### [](#pinning)Disable pinning in shared environments If you don’t enable CPU pinning and Redpanda runs in a shared environment, you can make Redpanda less aggressive in CPU usage by enabling overprovisioning. This adjustment ensures a fairer distribution of CPU time among all processes, improving overall system efficiency at the cost of Redpanda’s performance. You can enable overprovisioning by either setting the CPU request to a fractional value or setting `overprovisioned` to `true`. > ⚠️ **CAUTION** > > When `resources.cpu.cores` is set to a fractional value, such as `1100m` or `1.1`, the Pod is not eligible for exclusive access to the cores which leads to performance bottlenecks. #### Helm + Operator `redpanda-cluster.yaml` ```yaml apiVersion: cluster.redpanda.com/v1alpha1 kind: Redpanda metadata: name: redpanda spec: chartRef: {} clusterSpec: resources: cpu: cores: overprovisioned: true ``` ```bash kubectl apply -f redpanda-cluster.yaml --namespace ``` #### Helm ##### --values `cpu-cores-overprovisioned.yaml` ```yaml resources: cpu: cores: overprovisioned: true ``` ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --values cpu-cores-overprovisioned.yaml --reuse-values ``` ##### --set ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --set resources.cpu.cores= \ --set resources.cpu.overprovisioned=true ``` If you’re experimenting with Redpanda in Kubernetes, you can also set the number of CPU cores to millicores to automatically enable overprovisioning. `cpu-cores.yaml` ```yaml resources: cpu: cores: 200m ``` ## [](#memory)Configure memory resources On a worker node, Kubernetes and Redpanda processes are running at the same time, including the Seastar subsystem that is built into the Redpanda binary. Each of these processes consumes memory. You can configure the memory resources that are allocated to these processes. By default, the Helm chart allocates 80% of the configured memory in `resources.memory.container` to Redpanda, with the remaining reserved for overhead such as the Seastar subsystem and other container processes. Redpanda Data recommends this default setting. > 📝 **NOTE** > > Although you can also allocate the exact amount of memory for Redpanda and the Seastar subsystem manually, Redpanda Data does not recommend this approach because setting the wrong values can lead to performance issues, instability, or data loss. As a result, this approach is not documented here. ### Helm + Operator `redpanda-cluster.yaml` ```yaml apiVersion: cluster.redpanda.com/v1alpha1 kind: Redpanda metadata: name: redpanda spec: chartRef: {} clusterSpec: resources: memory: enable_memory_locking: true (1) container: # If omitted, the `min` value is equal to the `max` value (requested resources defaults to limits) # min: max: (2) ``` ```bash kubectl apply -f redpanda-cluster.yaml --namespace ``` ### Helm #### --values `memory.yaml` ```yaml resources: memory: enable_memory_locking: true (1) container: # If omitted, the `min` value is equal to the `max` value (requested resources defaults to limits) # min: max: (2) ``` ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --values memory.yaml --reuse-values ``` #### --set ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --set resources.memory.enable_memory_locking=true \ (1) --set resources.memory.container.max= (2) ``` | 1 | For production, enable memory locking to prevent the operating system from paging out Redpanda’s memory to disk, which can significantly impact performance. | | --- | --- | | 2 | The amount of memory to give Redpanda, Seastar, and the other container processes. You should give Redpanda at least 2 Gi of memory per core. Given that the Helm chart allocates 80% of the container’s memory to Redpanda, leaving the rest for the Seastar subsystem and other processes, set this value to at least 2.5 Gi per core to ensure Redpanda has a full 2 Gi. Redpanda supports the following memory resource units: B, K, M, G, Ki, Mi, and Gi. Memory units are converted to the nearest whole MiB. For a description of memory resource units, see the Kubernetes documentation. | ## [](#qos)Quality of service and resource guarantees To ensure that Redpanda receives stable and consistent resources, set the [quality of service (QoS) class](https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/#create-a-pod-that-gets-assigned-a-qos-class-of-guaranteed) to `Guaranteed` by matching resource requests and limits on all containers in the Pods that run Redpanda. Kubernetes uses QoS to decide which Pods to evict from a worker node that runs out of resources. When a worker node runs out of resources, Kubernetes evicts Pods with a `Guaranteed` QoS last. This stability is crucial for Redpanda because it requires consistent computational and memory resources to maintain high performance. Kubernetes gives a Pod a `Guaranteed` QoS class when every container inside it has identical resource requests and limits set for both CPU and memory. This strict configuration signals to Kubernetes that these resources are critical and should not be throttled or reclaimed under normal operating conditions. To configure the Pods that run Redpanda with Guaranteed QoS, specify both resource requests and limits for all _enabled_ containers in the Pods, including For default values and documentation for configuration options, see the [`values.yaml`](https://artifacthub.io/packages/helm/redpanda-data/redpanda?modal=values&path=statefulset.sideCars) file. and For default values and documentation for configuration options, see the [`values.yaml`](https://artifacthub.io/packages/helm/redpanda-data/redpanda?modal=values&path=statefulset.initContainers) file.. For example: ### Helm + Operator `redpanda-cluster.yaml` ```yaml apiVersion: cluster.redpanda.com/v1alpha1 kind: Redpanda metadata: name: redpanda spec: chartRef: {} clusterSpec: resources: memory: container: min: max: statefulset: sideCars: configWatcher: resources: requests: cpu: memory: limits: cpu: # Matches the request memory: # Matches the request initContainers: tuning: resources: requests: cpu: memory: limits: cpu: # Matches the request memory: # Matches the request setTieredStorageCacheDirOwnership: resources: requests: cpu: memory: limits: cpu: # Matches the request memory: # Matches the request configurator: resources: requests: cpu: memory: limits: cpu: # Matches the request memory: # Matches the request ``` ```bash kubectl apply -f redpanda-cluster.yaml --namespace ``` ### Helm #### --values `memory.yaml` ```yaml resources: memory: container: min: max: statefulset: sideCars: configWatcher: resources: requests: cpu: memory: limits: cpu: # Matches the request memory: # Matches the request initContainers: tuning: resources: requests: cpu: memory: limits: cpu: # Matches the request memory: # Matches the request setTieredStorageCacheDirOwnership: resources: requests: cpu: memory: limits: cpu: # Matches the request memory: # Matches the request configurator: resources: requests: cpu: memory: limits: cpu: # Matches the request memory: # Matches the request ``` ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --values memory.yaml --reuse-values ``` #### --set ```bash helm upgrade --install redpanda redpanda/redpanda --namespace --create-namespace \ --set resources.memory.container.min= \ --set resources.memory.container.max= \ --set statefulset.sideCars.configWatcher.resources.requests.cpu= \ --set statefulset.sideCars.configWatcher.resources.requests.memory= \ --set statefulset.sideCars.configWatcher.resources.limits.cpu= \ --set statefulset.sideCars.configWatcher.resources.limits.memory= \ --set statefulset.initContainers.tuning.resources.requests.cpu= \ --set statefulset.initContainers.tuning.resources.requests.memory= \ --set statefulset.initContainers.tuning.resources.limits.cpu= \ --set statefulset.initContainers.tuning.resources.limits.memory= \ --set statefulset.initContainers.setTieredStorageCacheDirOwnership.resources.requests.cpu= \ --set statefulset.initContainers.setTieredStorageCacheDirOwnership.resources.requests.memory= \ --set statefulset.initContainers.setTieredStorageCacheDirOwnership.resources.limits.cpu= \ --set statefulset.initContainers.setTieredStorageCacheDirOwnership.resources.limits.memory= \ --set statefulset.initContainers.configurator.resources.requests.cpu= \ --set statefulset.initContainers.configurator.resources.requests.memory= \ --set statefulset.initContainers.configurator.resources.limits.cpu= \ --set statefulset.initContainers.configurator.resources.limits.memory= ``` When the StatefulSet is deployed, make sure that the QoS for the Pods is set to `Guaranteed`: ```bash kubectl --namespace= get pod -o jsonpath='{ .status.qosClass}{"\n"}' ``` ## [](#configure-storage-capacity)Configure storage capacity Make sure to provision enough disk storage for your streaming workloads. If you use PersistentVolumes, you can set the storage capacity for each volume. For instructions, see [Configure Storage for the Redpanda data directory in Kubernetes](https://docs.redpanda.com/streaming/23.3/manage/kubernetes/storage/k-configure-storage/). ## [](#suggested-reading)Suggested reading - [Redpanda Helm Specification](https://docs.redpanda.com/streaming/23.3/reference/k-redpanda-helm-spec/#resources) - [Redpanda CRD Reference](https://docs.redpanda.com/streaming/23.3/reference/k-crd/) ## Suggested labs - [Redpanda Iceberg Docker Compose Example](https://docs.redpanda.com/labs/docker-compose/iceberg/) - [Enable Unified Identity with Azure Entra ID for Redpanda and Redpanda Console](https://docs.redpanda.com/labs/docker-compose/oidc/) - [Owl Shop Example Application in Docker](https://docs.redpanda.com/labs/docker-compose/owl-shop/) - [Migrate Data with Redpanda Migrator](https://docs.redpanda.com/labs/docker-compose/redpanda-migrator/) - [Start a Single Redpanda Broker with Redpanda Console in Docker](https://docs.redpanda.com/labs/docker-compose/single-broker/) - [Start a Cluster of Redpanda Brokers with Redpanda Console in Docker](https://docs.redpanda.com/labs/docker-compose/three-brokers/) - [Iceberg Streaming on Kubernetes with Redpanda, MinIO, and Spark](https://docs.redpanda.com/labs/kubernetes/iceberg/) See more [Search all labs](https://docs.redpanda.com/labs)