Delete PersistentVolumes in Kubernetes

Deleting a PersistentVolume (PV) can be necessary for a variety of reasons, such as removing a broker and its data from the cluster or performing maintenance or upgrades on the PersistentVolume.

Prerequisites

  • A running Redpanda deployment on a Kubernetes cluster.

  • PersistentVolumes for either the Redpanda data directory or the Tiered Storage cache.

Delete a PersistentVolume

To delete a PersistentVolume, follow these steps to ensure that your data is moved to other brokers in the cluster.

  1. Identify the PV that you want to delete:

    kubectl get persistentvolume

  2. Decommission the broker that has a PersistentVolumeClaim (PVC) bound to the PV.

    Decommissioning ensures no data loss by gracefully moving the broker’s topic partitions and replicas to other brokers in the cluster.

  3. Delete the PVC that is bound to your PV:

    kubectl delete persistentvolumeclaim <pvc-name> --namespace <namespace>

    To prevent accidental loss of data, PersistentVolumesClaims are not deleted when Redpanda brokers are removed from a cluster. When you no longer need PersistentVolumeClaims, you must delete them manually. Check the reclaim policy of your PersistentVolumes before deleting a PersistentVolumeClaim.

    kubectl get persistentvolume --namespace <namespace>

  4. If the reclaimPolicy of your PV is not Delete, delete the PV:

    kubectl delete persistentvolume <pv-name>

  5. Delete the Pod whose PVC was bound to the deleted PV:

    The StatefulSet schedules a new Pod on the same worker node and assigns it a unique node ID.

    If you use PVs for the Redpanda data directory, the Pod will have a new PVC bound to a PV that is set in storage.persistentVolume.storageClass. See Use PersistentVolumes.

    If you use PVs for the Tiered Storage cache, the Pod will have a new PVC bound to a PV that is set in storage.tieredStoragePersistentVolume.storageClass. See Tiered Storage Caching.

  6. Verify that the new Redpanda broker is running and that it has access to the appropriate PersistentVolume.

    kubectl --namespace <namespace> exec -ti <pod-name> -c <container-name> -- \
  rpk cluster info

You should see your new broker running with a new node ID.

Suggested reading