Docs Self-Managed Manage Kubernetes Networking and Connectivity Connect to Redpanda You are viewing the Self-Managed v24.3 beta documentation. We welcome your feedback at the Redpanda Community Slack #beta-feedback channel. To view the latest available version of the docs, see v24.2. Connect to Redpanda in Kubernetes To work with a Redpanda cluster running in Kubernetes, you can connect clients to the listeners that are exposed by Redpanda brokers. Depending on how the listeners are configured, you can access them from within the Kubernetes cluster and/or from outside the Kubernetes cluster. API Purpose Admin API Operate Redpanda clusters. For example, you can modify the cluster’s configuration, decommission brokers, and place brokers in maintenance mode. Kafka API Interact with the Kafka protocol in Redpanda. HTTP Proxy (PandaProxy) Access your data through a REST API. For example, you can list topics or brokers, get events, and produce events. Schema Registry Store and manage event schemas. For example, you can query supported serialization formats, register schemas for a subject, and retrieve schemas of specific versions. Prerequisites You must have the following: Kubernetes cluster: Ensure you have a running Kubernetes cluster, either locally, such as with minikube or kind, or remotely. Kubectl: Ensure you have the kubectl command-line tool installed and configured to communicate with your cluster. Redpanda cluster: Deploy a Redpanda cluster. Connect to an internal cluster To connect a client to Redpanda brokers running in the same Kubernetes cluster, use their fully qualified domain names (FQDNs) and the internal port of a listener. Together, the FQDN and internal port are called an endpoint. These endpoints may be secured using TLS and/or authentication. The rpk client on each Redpanda broker is pre-configured to connect to the internal Admin API and internal Kafka API of the local Redpanda cluster. To use other clients, such as a Kafka client, you must configure them. Connect internally with a local rpk client The rpk command-line client, available on each Redpanda broker, allows you to communicate with the internal listeners of both the Admin API endpoint and the Kafka API endpoint. By default, the Redpanda Helm chart configures rpk with a local redpanda.yaml configuration file located in the /etc/redpanda/ directory. As a result, you can use rpk from inside the container. For example, this command executes the rpk cluster info command: kubectl exec <pod-name> --namespace <namespace> -- rpk cluster info If you have SASL authentication enabled for the internal Kafka API listeners, you must specify the username, password, and SASL mechanism. kubectl exec <pod-name> --namespace <namespace> -- rpk cluster info \ -X user=<username> -X pass=<password> -X sasl.mechanism=<mechanism> You can also use environment variables instead of flags to specify the username, password, and SASL mechanism: export RPK_USER=<username> export RPK_PASS=<password> export RPK_SASL_MECHANISM=<mechanism> For details about SASL authentication, see Configure Authentication for Redpanda in Kubernetes. Connect to the internal Kafka API To connect to the internal Kafka API using a client other than rpk, you must configure the client with the correct endpoints, authentication credentials, and TLS certificates. You can find the connection details in the /etc/redpanda/redpanda.yaml file of any Pod that’s running a Redpanda broker: kubectl exec <pod-name> --namespace <namespace> -- cat /etc/redpanda/redpanda.yaml The rpk.kafka_api.brokers list contains the internal Kafka API endpoints of the Redpanda brokers: redpanda.yaml rpk: kafka_api: brokers: - redpanda-0.redpanda.redpanda.svc.cluster.local.:9093 - redpanda-1.redpanda.redpanda.svc.cluster.local.:9093 - redpanda-2.redpanda.redpanda.svc.cluster.local.:9093 If the internal listeners have SASL authentication enabled, you must also configure your clients with valid credentials. To find out if a listener has authentication enabled, check the Helm values: helm get values <release-name> --namespace <namespace> --all In this example, the Kafka API has SASL authentication enabled: auth: sasl: enabled: true listeners: kafka: port: 9093 # default is "sasl" when empty or "null" authenticationMethod: null For details about SASL authentication, see Configure Authentication for Redpanda in Kubernetes. If the internal listeners have TLS or mTLS enabled, you must configure your clients with valid TLS files. To find out if the Redpanda cluster has TLS or mTLS enabled, check the Helm values. In this example, the Kafka API has TLS and mTLS enabled: listeners: kafka: port: 9093 tls: cert: default requireClientAuth: true TLS files are stored in Secret resources that you can mount onto any Pods that run clients. TLS files may include: Certificate files (*.crt): These files contain the public key and the identity (domain name) and are used for encryption. They can be self-signed or signed by a certificate authority (CA). Key files (*.key): These contain the private key associated with the certificate. The private key should be kept secure and confidential. CA files: These are certificates of the certificate authorities. They are used to verify if a given certificate is trusted. You can find the names of all TLS Secrets using this command: join -t $'\t' \ <(kubectl get pod <pod-name> --namespace <namespace> -o jsonpath="{range .spec.containers[0].volumeMounts[*]}{.name}{'\t'}{.mountPath}{'\n'}{end}" | awk '$2 ~ /^\/etc\/tls\/certs\// {print $1"\t"$2}' | sort) \ <(kubectl get pod <pod-name> --namespace <namespace> -o jsonpath="{range .spec.volumes[?(@.secret)]}{.name}{'\t'}{.secret.secretName}{'\n'}{end}" | sort) \ | awk 'BEGIN{printf "%-25s\t%-40s\n", "SECRET", "MOUNT PATH"} {printf "%-25s\t%-40s\n", $3, $2}' For example: SECRET MOUNT PATH redpanda-client /etc/tls/certs/redpanda-client redpanda-default-cert /etc/tls/certs/default redpanda-external-cert /etc/tls/certs/external Then, you can mount the required Secrets into the Pods that run the clients: apiVersion: v1 kind: Pod metadata: name: redpanda-client-pod labels: app: redpanda-client spec: volumes: - name: tls-certs secret: secretName: redpanda-client containers: - name: client-container image: example/client-image volumeMounts: - name: tls-certs mountPath: /etc/tls/certs readOnly: true Now, you can configure clients with the mount path to the TLS files in your Secrets. For details about TLS, see TLS for Redpanda in Kubernetes. Connect to the internal HTTP Proxy To connect to the HTTP Proxy, use its configured internal port. To find the port, check the Helm values: helm get values <release-name> --namespace <namespace> --all In this example, the internal port is 8082. listeners: http: port: 8082 To test an internal connection, you can use the cURL command-line client inside the container running a Redpanda broker: kubectl exec <pod-name> --namespace <namespace> -- curl http://redpanda-0.redpanda.redpanda.svc.cluster.local:8082/topics -sS If SASL authentication is enabled, provide a valid username and password using basic authentication: kubectl exec <pod-name> --namespace <namespace> -- curl http://redpanda-0.redpanda.redpanda.svc.cluster.local:8082/topics -u <username>:<password> -sS If TLS is enabled, specify the HTTPS protocol and pass the path to the ca.crt file: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:8082/topics --cacert /etc/tls/certs/default/ca.crt -sS If the broker’s certificate is signed by a well-known, trusted CA, and you’re confident about the integrity of your system’s CA trust store, you don’t need the --cacert flag. If mTLS is enabled, pass the path to the client’s key and certificate: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:8082/topics \ --cacert /etc/tls/certs/default/ca.crt \ --cert /etc/tls/certs/redpanda-client/tls.crt \ --key /etc/tls/certs/redpanda-client/tls.key For all available endpoints, see HTTP Proxy API. Connect to internal Schema Registry To connect to the Schema Registry, use its configured internal port. To find the port, check the Helm values: helm get values <release-name> --namespace <namespace> --all In this example, the internal port is 8081. listeners: schemaRegistry: port: 8081 To test an internal connection, you can use the cURL command-line client inside the container running a Redpanda broker: kubectl exec <pod-name> --namespace <namespace> -- curl http://redpanda-0.redpanda.redpanda.svc.cluster.local:8081/subjects -sS If SASL authentication is enabled, provide a username and password using basic authentication: kubectl exec <pod-name> --namespace <namespace> -- curl http://redpanda-0.redpanda.redpanda.svc.cluster.local:8081/subjects -u <username>:<password> -sS If TLS is enabled, specify the HTTPS protocol and pass the path to the ca.crt file: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:8081/subjects --cacert /etc/tls/certs/default/ca.crt -sS If the broker’s certificate is signed by a well-known, trusted CA, and you’re confident about the integrity of your system’s CA trust store, you don’t need the --cacert flag. If mTLS is enabled, pass the path to the client’s key and certificate: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:8081/subjects \ --cacert /etc/tls/certs/default/ca.crt \ --cert /path/to/client.crt \ --key /path/to/client.key For all available endpoints, see Schema Registry API. Connect to the internal Admin API To connect to the Admin API, use its configured internal port. To find the port, check the Helm values: helm get values <release-name> --namespace <namespace> --all In this example, the internal port is 8081. listeners: admin: port: 9644 To test an internal connection, you can use the cURL command-line client inside the container running a Redpanda broker: kubectl exec <pod-name> --namespace <namespace> -- curl http://redpanda-0.redpanda.redpanda.svc.cluster.local:9644/v1/node_config -sS If TLS is enabled, specify the HTTPS protocol and pass the path to the ca.crt file: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:9644/v1/node_config --cacert /etc/tls/certs/default/ca.crt -sS If the broker’s certificate is signed by a well-known, trusted CA, and you’re confident about the integrity of your system’s CA trust store, you don’t need the --cacert flag. If mTLS is enabled, pass the path to the client’s key and certificate: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:9644/v1/node_config \ --cacert /etc/tls/certs/default/ca.crt \ --cert /path/to/client.crt \ --key /path/to/client.key For all available endpoints, see Admin API. Connect to an external cluster To connect to your Redpanda cluster from outside Kubernetes, the Redpanda cluster must be configured with external access. See Configure External Access. Create an rpk profile An rpk profile contains a reusable configuration for a Redpanda cluster. When running rpk, you can create a profile, configure it for a cluster you’re working with, and use it repeatedly when running an rpk command for the cluster. When external.enabled is set to true (default), the Helm chart generates a ConfigMap that contains settings for an rpk profile. You can use these settings to connect to the cluster externally. The ConfigMap configures an rpk profile using the listeners.admin.external.default and listeners.kafka.external.default objects in Helm values. Install rpk. Configure rpk to use the profile in the ConfigMap: rpk profile create --from-profile <(kubectl get configmap --namespace <namespace> redpanda-rpk -o go-template='{{ .data.profile }}') <profile-name> If you have SASL authentication enabled, you must configure rpk with a valid username and password. When you first deploy Redpanda, the Helm chart prints some notes with the commands necessary to configure a username and password locally. For example: kubectl --namespace <namespace> get secret <secret-name> -o go-template="{{ range .data }}{{ . | base64decode }}{{ end }}" | IFS=: read -r RPK_USER RPK_PASS RPK_SASL_MECHANISM export RPK_USER RPK_PASS RPK_SASL_MECHANISM If you have TLS or mTLS enabled, you must save the TLS files to your local filesystem external to the Kubernetes cluster. When you first deploy Redpanda, the Helm chart prints some notes with the commands necessary to save the TLS files locally. For example: kubectl get secret --namespace <namespace> <secret-name> -o go-template='{{ index .data "ca.crt" | base64decode }}' > ca.crt kubectl get secret --namespace <namespace> <secret-name> -o go-template='{{ index .data "tls.crt" | base64decode }}' > tls.crt kubectl get secret --namespace <namespace> <secret-name> -o go-template='{{ index .data "tls.key" | base64decode }}' > tls.key For more details about rpk profiles, see rpk Profiles. Connect to the external Kafka API To connect to the external Kafka API using a client other than rpk, you must configure the client with the correct broker endpoints, authentication credentials, and TLS certificates. You can find the connection details in the /etc/redpanda/redpanda.yaml file of any Pod that’s running a Redpanda broker: kubectl exec <pod-name> --namespace <namespace> -- cat /etc/redpanda/redpanda.yaml The redpanda.advertised_kafka_api list item called default contains the external Kafka API endpoints for the Redpanda brokers: redpanda.yaml redpanda: advertised_kafka_api: - address: redpanda-0.redpanda.redpanda.svc.cluster.local. port: 9093 name: internal - address: redpanda-0.customredpandadomain.local port: 31092 name: default If the external listeners have SASL authentication enabled, you must also configure your clients with valid credentials. To find out if the Redpanda cluster has authentication enabled, check the Helm values: helm get values <release-name> --namespace <namespace> --all In this example, the Kafka API has SASL authentication enabled: auth: sasl: enabled: true listeners: kafka: external: default: # default is "sasl" when empty or "null" authenticationMethod: null For details about SASL authentication, see Configure Authentication for Redpanda in Kubernetes. If the external listeners have TLS or mTLS enabled, you must configure your clients with valid TLS files. To find out if the Redpanda cluster has TLS enabled, check the Helm values. In this example, the Kafka API has TLS enabled: listeners: kafka: external: default: # enabled: true port: 9094 advertisedPorts: - 31092 tls: # enabled: true cert: external TLS files are stored in Secrets that you can mount onto the Pods that are running the clients. TLS files may include: Certificate files (*.crt): These files contain the public key and the identity (domain name) and are used for encryption. They can be self-signed or signed by a certificate authority (CA). Key files (*.key): These contain the private key associated with the certificate. The private key should be kept secure and confidential. CA files: These are certificates of the certificate authorities. They are used to verify if a given certificate is trusted. You can find the names of all TLS Secrets using this command: join -t $'\t' \ <(kubectl get pod <pod-name> --namespace <namespace> -o jsonpath="{range .spec.containers[0].volumeMounts[*]}{.name}{'\t'}{.mountPath}{'\n'}{end}" | awk '$2 ~ /^\/etc\/tls\/certs\// {print $1"\t"$2}' | sort) \ <(kubectl get pod <pod-name> --namespace <namespace> -o jsonpath="{range .spec.volumes[?(@.secret)]}{.name}{'\t'}{.secret.secretName}{'\n'}{end}" | sort) \ | awk 'BEGIN{printf "%-25s\t%-40s\n", "SECRET", "MOUNT PATH"} {printf "%-25s\t%-40s\n", $3, $2}' SECRET MOUNT PATH redpanda-client /etc/tls/certs/redpanda-client redpanda-default-cert /etc/tls/certs/default redpanda-external-cert /etc/tls/certs/external Then, you can save the TLS files to your local file system. For example: kubectl get secret --namespace <namespace> redpanda-client -o go-template='{{ index .data "ca.crt" | base64decode }}' > ca.crt kubectl get secret --namespace <namespace> redpanda-client -o go-template='{{ index .data "tls.crt" | base64decode }}' > tls.crt kubectl get secret --namespace <namespace> redpanda-client -o go-template='{{ index .data "tls.key" | base64decode }}' > tls.key Now, you can configure clients with the path to the TLS files. For details about TLS, see TLS for Redpanda in Kubernetes. Connect to the external HTTP Proxy To connect to the HTTP Proxy, use its configured external port. To find the port, check the Helm values: helm get values <release-name> --namespace <namespace> --all In this example, the external port on the container is 8082. The external node port on the worker node is 30082. listeners: http: external: default: port: 8083 advertisedPorts: - 30082 To test an external connection, you can use the cURL command-line client inside the container running a Redpanda broker: kubectl exec <pod-name> --namespace <namespace> -- curl http://redpanda-0.redpanda.redpanda.svc.cluster.local:8083/topics -sS If SASL authentication is enabled, provide a username and password using basic authentication: kubectl exec <pod-name> --namespace <namespace> -- curl http://redpanda-0.redpanda.redpanda.svc.cluster.local:8083/topics -u <username>:<password> -sS If TLS is enabled, specify the HTTPS protocol and pass the path to the ca.crt file: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:8083/topics --cacert /etc/tls/certs/external/ca.crt -sS If the broker’s certificate is signed by a well-known, trusted CA, and you’re confident about the integrity of your system’s CA trust store, you don’t need the --cacert flag. If mTLS is enabled, pass the path to the client’s key and certificate: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:8083/topics \ --cacert /etc/tls/certs/external/ca.crt \ --cert /etc/tls/certs/external/tls.crt \ --key /etc/tls/certs/external/tls.key For all available endpoints, see HTTP Proxy API. Connect to external Schema Registry To connect to the Schema Registry, use its configured external port. To find the port, check the Helm values: helm get values <release-name> --namespace <namespace> --all In this example, the external port on the container is 8084. The external node port on the worker node is 30081. listeners: schemaRegistry: external: default: port: 8084 advertisedPorts: - 30081 To test an external connection, you can use the cURL command-line client inside the container running a Redpanda broker: kubectl exec <pod-name> --namespace <namespace> -- curl http://redpanda-0.redpanda.redpanda.svc.cluster.local:8084/subjects -sS If SASL authentication is enabled, provide a username and password using basic authentication: kubectl exec <pod-name> --namespace <namespace> -- curl http://redpanda-0.redpanda.redpanda.svc.cluster.local:8084/subjects -u <username>:<password> -sS If TLS is enabled, specify the HTTPS protocol and pass the path to the ca.crt file: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:8084/subjects --cacert /etc/tls/certs/external/ca.crt -sS If the broker’s certificate is signed by a well-known, trusted CA, and you’re confident about the integrity of your system’s CA trust store, you don’t need the --cacert flag. If mTLS is enabled, pass the path to the client’s key and certificate: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:8084/subjects \ --cacert /etc/tls/certs/external/ca.crt \ --cert /etc/tls/certs/external/tls.crt \ --key /etc/tls/certs/external/tls.key For all available endpoints, see Schema Registry API. Connect to external Admin API To connect to the Admin API, use its configured external port. To find the port, check the Helm values: helm get values <release-name> --namespace <namespace> --all In this example, the external port on the container is 8084. The external node port on the worker node is 30081. listeners: schemaRegistry: external: default: port: 9645 advertisedPorts: - 31644 To test an external connection, you can use the cURL command-line client inside the container running a Redpanda broker: kubectl exec <pod-name> --namespace <namespace> -- curl http://redpanda-0.redpanda.redpanda.svc.cluster.local:9645/v1/node_config -sS If TLS is enabled, specify the HTTPS protocol and pass the path to the ca.crt file: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:9645/v1/node_config --cacert /etc/tls/certs/external/ca.crt -sS If the broker’s certificate is signed by a well-known, trusted CA, and you’re confident about the integrity of your system’s CA trust store, you don’t need the --cacert flag. If mTLS is enabled, pass the path to the client’s key and certificate: kubectl exec <pod-name> --namespace <namespace> -- curl https://redpanda-0.redpanda.redpanda.svc.cluster.local:9645/v1/node_config \ --cacert /etc/tls/certs/external/ca.crt \ --cert /etc/tls/certs/external/tls.crt \ --key /etc/tls/certs/external/tls.key For all available endpoints, see Admin API. Next steps Configure Listeners in Kubernetes Suggested reading About Networking and Connectivity in Kubernetes rpk Profiles Configure Authentication for Redpanda in Kubernetes TLS for Redpanda in Kubernetes API Reference Kubernetes Helm Chart Specifications Kubernetes Custom Resource Definitions Back to top × Simple online edits For simple changes, such as fixing a typo, you can edit the content directly on GitHub. Edit on GitHub Or, open an issue to let us know about something that you want us to change. Open an issue Contribution guide For extensive content updates, or if you prefer to work locally, read our contribution guide . Was this helpful? thumb_up thumb_down group Ask in the community mail Share your feedback group_add Make a contribution Overview Configure Listeners