Configuring Redpanda mTLS on Kubernetes
This section provides details on how to enable mutual Transport Layer Security (mTLS) authentication on Kubernetes. mTLS authenticates the server and encrypts communication between the server and client, as one-way TLS does, and also authenticates the client. Redpanda supports mTLS authentication on all four APIs.
For general security information, see Redpanda Security on Kubernetes.
Prerequisites
-
Configure the kubectl context.
-
Install cert-manager.
-
Install the Redpanda operator.
If you haven’t completed these tasks, follow one of the Install Redpanda Guide for Kubernetes to get set up before you configure TLS. Just stop before you install and connect to the Redpanda cluster.
Step 1: Create the cluster specification file
The Redpanda GitHub repository contains an mtls.yaml
sample configuration file you can use to enable mTLS authentication: redpanda-examples/docs/example-config/kubernetes/mtls.yaml.
To modify the sample file, save it locally and make the required changes. |
The text below is the Redpanda mtls.yaml
file with the relevant sections highlighted:
apiVersion: redpanda.vectorized.io/v1alpha1
kind: Cluster
metadata:
name: cluster-sample-mtls
spec:
image: "vectorized/redpanda"
version: "latest"
replicas: 1
resources:
requests:
cpu: 1
memory: 1.2G
limits:
cpu: 1
memory: 1.2G
configuration:
rpcServer:
port: 33145
kafkaApi:
- port: 9092
tls:
enabled: true
requireClientAuth: true
pandaproxyApi:
- port: 8082
tls:
enabled: true
requireClientAuth: true
schemaRegistry:
port: 8081
tls:
enabled: true
requireClientAuth: true
adminApi:
- port: 9644
tls:
enabled: true
requireClientAuth: true
developerMode: true
Complete these steps to configure the cluster specification file. The steps reference the highlighted lines in the file:
-
Configure the cluster name. The following property in the file is the name of the cluster. Change this value to the name of your cluster:
name: cluster-sample-mtls
-
Enable mTLS for each API individually. You can enable mTLS on one or more of the APIs. In the example, all four APIs have mTLS enabled:
kafkaApi: - port: 9092 tls: enabled: true requireClientAuth: true pandaproxyApi: - port: 8082 tls: enabled: true requireClientAuth: true schemaRegistry: port: 8081 tls: enabled: true requireClientAuth: true adminApi: - port: 9644 tls: enabled: true requireClientAuth: true
Step 2: Configure external connectivity (optional)
The table below lists the supported listener configurations for each API with mTLS enabled. See Listeners.
API | Listener configurations with mTLS |
---|---|
Kafka API |
<ul><li> One internal listener with mTLS enabled </li><li> One internal listener and one external listener. Only one of the listeners can have mTLS enabled. </li> </ul> |
Schema Registry |
<ul><li> One internal listener with mTLS enabled </li><li> One listener that is used for internal and external connectivity with mTLS enabled </li> </ul> |
You can specify up to two listeners for each API, but only one listener can have mTLS enabled. If you do have two listeners, one must be internal, and one must be external. The exception is Schema Registry. The Schema Registry listener can be internal, or it can be an internal port that is used internally and externally. If you enable external connectivity on Schema Registry, the Kubernetes node port connects to the internal Redpanda port to provide external connectivity.
To enable external connectivity with mTLS, add the following lines to each API in the configuration file that you created in Step 1:
- external:
enabled: true
subdomain: <subdomain_name>
The subdomain
field lets you specify the advertised address of the external listener. The subdomain addresses, including the brokers, must be registered with a DNS provider, such as Amazon Route 53. You only need to include the subdomain name in this field, not the brokers. Each API in the configuration file must have the same subdomain
specified.
The external port is generated automatically, and you do not need to specify it. In the following example, mTLS is enabled on the external listener for the Kafka API. Enable external connectivity the same way for Admin API and HTTP Proxy.
kafkaApi:
- port: 9092
- external:
enabled: true
subdomain: <subdomain_name>
tls:
enabled: true
requireClientAuth: true
The Schema Registry syntax is slightly different, in that the ports are not a list. You can specify one internal port and one external port. Schema Registry always uses an internal port and with external connectivity configured, the Kubernetes node port connects to the internal Redpanda port. Configure mTLS with external connectivity for Schema Registry like this:
schemaRegistry:
port: 8081
external:
enabled: true
subdomain: <subdomain_name>
tls:
enabled: true
requireClientAuth: true
For information about external connectivity, including subdomains, see External connectivity.
Step 3: Provide an issuer or certificate (optional)
Kafka API and Schema Registry let you provide a certificate issuer or certificate for the node certificate.
When you enable mTLS, the Redpanda operator generates a root certificate for each API. The root certificate is local to the cluster and the operator uses the root certificate to generate leaf certificates for the nodes and the client. However, for Kafka API and Schema registry you can instead specify a certificate issuer or a certificate.
For information about how certificates are created and used in Redpanda, see Certificates.
Provide an issuer
To provide a certificate issuer, add the issuerRef
property to the cluster specification file that you created in the previous step. For information about issuers, see the cert-manager Issuer documentation.
You can provide an issuer for kafkaAPI
or schemaRegistry
in the same way. The example here is the kafkaAPI
configuration configuration with the issuerRef
property highlighted:
kafkaApi:
- port: 9092
tls:
enabled: true
issuerRef:
name: <issuer_name>
kind: <issuer>
requireClientAuth: true
The issuerRef
property contains the following variables:
Variable | Description |
---|---|
|
The name of the issuer or cluster issuer. |
|
A Kubernetes resource that represents a certificate authority. The value of this property can be |
Provide a certificate
You can provide a certificate as a secret by adding the nodeSecretRef
property to the cluster specification file that you created. For information about Secrets, see the Kubernetes Secrets documentation. The cert-manager Certificate documentation contains detailed information about certificates, including a diagram of the certificate lifecycle.
You can provide a certificate for kafkaAPI
or schemaRegistry
in the same way. The following example is the kafkaAPI
configuration with the nodeSecretRef
property highlighted:
kafkaApi:
- port: 9092
tls:
enabled: true
nodeSecretRef:
name: <secret_name>
namespace: <secret_namespace>
requireClientAuth: true
The nodeSecretRef
property contains the following variables:
Variable | Description |
---|---|
|
Name of the certificate secret. |
|
Kubernetes namespace where the certificate secret is. If the secret is in a different namespace than the Redpanda cluster, the operator copies it to the namespace of the Redpanda cluster. |
Step 4: Create the Redpanda cluster
After you configure the cluster specification file, you must run the kubectl apply
command to create the cluster. You can run the command using a path to the cluster specification file on your local machine, or you can use the URL to the mtls.yaml
file.
If you modified the file in the previous step, you have the file saved locally. To create the Redpanda cluster, run:
kubectl apply -f <cluster_specification.yaml>
If you did not modify the example file, you can use the URL to the example file in GitHub to create the cluster:
kubectl apply -f https://raw.githubusercontent.com/redpanda-data/redpanda-examples/main/docs/example-config/kubernetes/mtls.yaml
Step 5: Create the ConfigMap
Create a YAML file to hold the configuration for mTLS, including the location of the public certificate. In the next step, you create the Pod, which consumes this ConfigMap. This lets you run rpk
commands with mTLS authentication.
For detailed information, see the Kubernetes ConfigMaps documentation.
-
Copy the text below and save it locally as a YAML file, such as
mtls_config_map.yaml
.
apiVersion: v1
kind: ConfigMap
metadata:
name: <ConfigMap_name>
data:
redpanda.yaml: |
redpanda:
rpk:
kafka_api:
brokers:
- <cluster_name>-0.<cluster_name>.default.svc.cluster.local:9092
tls:
key_file: <key_file_path>/tls.key
cert_file: <cert_file_path>/tls.crt
truststore_file: <truststore_file_path>/ca.crt
-
In the file you just saved, configure these variables:
Variable Description ConfigMap_name
Name of the ConfigMap. This can be any string. This is what you use to reference the ConfigMap in the next step when you configure the Pod.
cluster_name
Name of the Redpanda cluster you defined in the cluster specification file.
key_file_path
Directory where you want to mount the
tls.key
private client key. Generally this is/etc/tls/certs
.cert_file_path
Filename and directory where you want to mount the
tls.crt
private key. Generally this is/etc/tls/certs
.truststore_file_path
Directory where you want to mount the
ca.crt
file. Generally this is/etc/tls/certs/ca
. -
Save the file.
External connectivity
If you’re configuring mTLS with external connectivity, you must configure the brokers accordingly. Replace the brokers
property in the example file with this:
brokers:
- 0.<subdomain_name>.:<node_port>
Configure the following variables in the brokers
property:
Variable | Description |
---|---|
|
Name of the subdomain that you include in the cluster specification file in Step 1. |
|
Kafka API external port. Unless you include this in the cluster specification file, this port is autogenerated by Kubernetes. |
Step 6: Configure the Pod
The Pod is the process that consumes the ConfigMap you created. This Pod runs the Redpanda image for running rpk
, which is part of the Redpanda image.
For detailed information about Pods, see the Kubernetes Pods documentation.
-
Copy the text below and save it locally as a YAML file, such as
mtls_pod.yaml
.apiVersion: v1 kind: Pod metadata: name: <pod_name> spec: containers: - name: rpk image: 'vectorized/redpanda:<redpanda-version>' command: - /bin/bash - '-c' args: - sleep infinity volumeMounts: - mountPath: <key_file_path> name: <tls_volume_name> - mountPath: <truststore_file_path> name: <ca_volume_name> - mountPath: /etc/redpanda name: <rpk_volume_name> restartPolicy: Never volumes: - name: <tls_volume_name> secret: secretName: <cluster_name>-user-client - name: <ca_volume_name> secret: secretName: <cluster_name>-redpanda - name: <rpk_volume_name> configMap: name: <configMap_name>
-
In the file you just saved, configure these variables:
Variable Description pod_name
Name of the Pod. This is the Pod that runs
rpk
. This can be any string.args
Specifies what you want the Pod to do. You can run
rpk
commands here. This example uses thesleep infinity
argument, which tells the Pod to keep running so that you can run as manyrpk
commands as you want from the command line. -
Configure the
volumeMounts
properties. There are three: one fortls
, one forca
, and one forrpk
.-
tls
- The path and the name of thetls.crt
andtls.key
volume mount.Variable Description key_file_path
The same path that you specified in the
key_file_path
andcert_file_path
properties in the ConfigMap. Generally this is/etc/tls/certs
.tls_volume_name
Must match the
tls_volume_name
in thevolumeMounts
property. -
ca
- The path and the name of theca.crt
volume mount.Variable Description truststore_file_path
The same path that you specified in the
truststore_file_path
property in the ConfigMap. Generally this is/etc/tls/certs/ca
.ca_volume_name
This can be any string, but it must match the
ca_volume_name
in thevolumes
property of this file. -
rpk
- The path and the name of therpk
volume mount.Variable Description name
This can be any string, but it must match the
rpk_volume_name
in thevolumes
property of this file.
-
-
Configure the
volumes
properties. There are three: one fortls
, one forca
, and one forrpk
.-
tls
- The name and secret of thetls.crt
andtls.key
volume mount.Variable Description tls_volume_name
This must be the same as the
tls_volume_name
in thevolumeMounts
property of this file.cluster_name
Cluster name that you defined in the cluster specification file in Step 1. The
secretName
property specifies the name of the client secret. For the Kafka API, this is<cluster_name>-user-client
. -
ca
- The name and secret of theca.crt
volume mount.Variable Description name
This must be the same as the
ca_volume_name
in thevolumeMounts
property.cluster_name
Cluster name that you defined in the cluster specification file in Step 1. The
secretName
property specifies the name of the node secret. For the Kafka API, this is<cluster_name>-redpanda
. -
rpk
- The volume name and ConfigMap name of therpk
volume mount.Variable Description rpk_volume_name
This must match the
<rpk_volume_name>
in thevolumeMounts
property of this file.configMap_name
ConfigMap name you specified in the
name
property of the ConfigMap in the previous step.
-
-
Configure the
<redpanda-version>
variable. Add a Redpanda version, such asv21.11.11
. You can find all Redpanda version tags in the Redpanda Docker Hub repository. -
Save the file.
Step 8: Connect to Redpanda
Now that you have mTLS enabled and the Pod created, you can start using rpk
to interact with Redpanda. Note that each time you run an rpk
command, rpk
establishes a connection and authenticates the server and the server authenticates the client.
-
Create a topic with this command:
kubectl exec <pod_name> -- rpk topic create <topic_name>
Configure these variables:
Variable Description pod_name
Name of the Pod that you specified in the Pod configuration file.
topic_name
Name of the topic that you’re creating with this command.
-
To describe the topic, run:
kubectl exec <pod_name> -- rpk topic describe <topic_name>
You don’t need to specify the brokers in these commands, because they were defined in the ConfigMap. If you include brokers in the rpk commands, it overrides the brokers in the ConfigMap.
|
Step 9: Clean up
You can use rpk commands to start producing and consuming to your cluster.
When you’re ready to delete your cluster and configuration files, run:
kubectl delete -f <cluster_specification.yaml> -f <mtls_config_map.yaml> -f <mtls_pod.yaml>