Docs Self-Managed Manage Security Configure Kafka TLS Encryption This is documentation for Self-Managed v23.2, which is no longer supported. To view the latest available version of the docs, see v24.2. Configure Kafka TLS Encryption By default, Redpanda data is sent unencrypted. A security best practice is to enable encryption with TLS or mTLS. Transport Layer Security (TLS), previously SSL, provides encryption for client-server communication. A server certificate prevents third parties from accessing data transferred between the client and server. mTLS, or 2-way TLS, is a protocol that authenticates both the server and the client. In addition to the server certificate required in TLS, mTLS also requires the client to give a certificate. This involves more overhead to implement, but it can be useful for environments that require additional security and only have a small number of verified clients. For each Redpanda broker, specify the key file (broker.key), the certificate file (broker.crt), and the truststore file (ca.crt). Each broker has its own broker.key and broker.crt, but all brokers can have the same ca.crt. Prerequisites TLS certificates are necessary for encryption. You can use your own certificates, either self-signed or issued by a trusted certificate authority (CA). Create a local CA for self-signed certificates This step is required if you want to generate multiple certificates all signed by the same root; for example, you want to use mTLS but issue different certificates to multiple Redpanda brokers and clients. To generate a self-signed certificate in a single command: openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 -keyout broker.key -out broker.crt -subj "/CN=redpanda" -addext "subjectAltName = DNS:localhost, IP: 127.0.0.1" 1. Create CA configuration file (ca.cnf) Edit the distinguished_name section with your own organization details. For default_md, sha256 is the minimum message digest level. The subjectAltName must be accurate for the broker’s certificate. ca.cnf [ ca ] default_ca = CA_default [ CA_default ] default_days = 365 database = index.txt serial = serial.txt default_md = sha256 copy_extensions = copy unique_subject = no policy = signing_policy [ signing_policy ] organizationName = supplied commonName = optional # Used to create the CA certificate. [ req ] prompt = no distinguished_name = distinguished_name x509_extensions = extensions [ distinguished_name ] organizationName = Redpanda commonName = Redpanda CA [ extensions ] keyUsage = critical,digitalSignature,nonRepudiation,keyEncipherment,keyCertSign basicConstraints = critical,CA:true,pathlen:1 # Common policy for nodes and users. [ signing_policy ] organizationName = supplied commonName = optional Create a CA key to self-sign certificates: openssl genrsa -out ca.key 2048 chmod 400 ca.key Create a public certificate for the CA: openssl req -new -x509 -config ca.cnf -key ca.key -days 365 -batch -out ca.crt where: Inputs Description -new New request. -x509 Create an X.509 certificate, instead of a certificate signing request (CSR). -config ca.cnf Configuration file to use when generating certificates (created above). -key ca.key Private key of the CA (created above). -days 365 Number of days signed certificates are valid. -batch Batch mode, where certificates are certified automatically. Output Description -out ca.crt The public key certificate of the CA. Create certificate signing requests This step creates the certificate signing request for the CA to extend trust over the broker’s certificates. 1. Create broker configuration file (broker.cnf) A subject alternative name (SAN) indicates all domain names and IP addresses secured by the certificate. Depending on the address the client uses to connect to Redpanda, you might need to create a CNF file for each broker to modify the alt_names section with organizational details. For production usage, edit alt_names with DNS resolutions and/or the IP addresses. broker.cnf [ req ] prompt = no distinguished_name = distinguished_name req_extensions = extensions [ distinguished_name ] organizationName = Redpanda [ extensions ] subjectAltName = @alt_names [ alt_names ] DNS.1 = localhost DNS.2 = redpanda DNS.3 = console DNS.4 = connect DNS.5 = ec2-3-15-15-272.us-east-2.compute.amazonaws.com IP.1 = 10.0.8.1 You could configure alternative names with a single version of broker.key/broker.crt, as long as you update the certificate for all brokers in the cluster any time you edit an entry. For example: [ alt_names ] DNS.1 = broker1.example.com DNS.2 = broker2.example.com DNS.3 = broker3.example.com Additionally, you can configure alternative names using the public or private IP addresses of all your brokers. For example: [ alt_names ] IP.1 = 10.0.8.1 IP.2 = 10.0.8.2 IP.3 = 10.0.8.3 2. Generate broker private key (broker.key) Generate a 2048-bit RSA private key for brokers: openssl genrsa -out broker.key 2048 where: Output Description -out broker.key The private key certificate for the broker. 3. Generate certificate signing request Before signing certificates, you must run the following command to generate the broker’s certificate signing request: openssl req -new -key broker.key -out broker.csr -nodes -config broker.cnf where: Inputs Description -req Input is a certificate request. Sign and output. -signkey ca.key Private key of the CA (created above). -days 365 Number of days signed certificates are valid. -extfile broker.cnf Configuration file for CA. -extensions extensions Section in broker.cnf to use when applying extensions. -in broker.csr Broker certificate signing request (CSR generated above). Output Description -out broker.crt The signed public key certificate for the broker. 4. Sign certificates Sign the certificate with the CA signature: touch index.txt echo '01' > serial.txt openssl ca -config ca.cnf -keyfile ca.key -cert ca.crt -extensions extensions -in broker.csr -out broker.crt -outdir . -batch chown redpanda:redpanda broker.key broker.crt ca.crt chmod 400 broker.key broker.crt ca.crt If generated by a corporate CA, these certificate signing requests must be signed with the following extensions: keyUsage = critical,digitalSignature,keyEncipherment extendedKeyUsage = serverAuth,clientAuth Configure TLS To configure TLS, in redpanda.yaml, enter: redpanda.yaml redpanda: rpc_server_tls: {} kafka_api: - address: 0.0.0.0 port: 9092 name: tls_listener kafka_api_tls: - name: tls_listener key_file: broker.key cert_file: broker.crt truststore_file: ca.crt enabled: true require_client_auth: false admin_api_tls: [] pandaproxy: pandaproxy_api_tls: [] schema_registry: schema_registry_api_tls: [] The following files must be readable by Redpanda, either through 444 permissions or chown to Redpanda with 400 permissions: broker.crt broker.key ca.crt Because the keys and certificates are only read at startup, you must restart Redpanda services after updating redpanda.yaml. TLS-related changes to redpanda.yaml will not be known to Redpanda until after this restart: systemctl restart redpanda To set the RPC port to encrypt replication, add: redpanda.yaml redpanda: rpc_server_tls: enabled: true require_client_auth: false key_file: broker.key cert_file: broker.crt truststore_file: ca.crt Schema Registry and HTTP Proxy connect to Redpanda over the Kafka API. If you configure a TLS listener for the Kafka API, you must add schema_registry_client::broker_tls and pandaproxy_client::broker_tls. All APIs, except the internal RPC port, support multiple listeners. See: Configure Schema Registry and HTTP Proxy to connect to Redpanda with SASL Configure Listeners Configure mTLS To enable mTLS, add require_client_auth set to true. For example, for the Kafka API, in redpanda.yaml, enter: redpanda.yaml redpanda: kafka_api: - address: 0.0.0.0 port: 9092 name: mtls_listener kafka_api_tls: - name: mtls_listener key_file: mtls_broker.key cert_file: mtls_broker.crt truststore_file: mtls_ca.crt enabled: true require_client_auth: true See also: Configure Listeners Configure mTLS for a Kafka API listener To enable mTLS for a Kafka API listener, edit redpanda.yaml: redpanda.yaml redpanda: kafka_api: - name: internal address: 0.0.0.0 port: 9092 advertised_kafka_api: - name: internal address: <port-clients-connect-to> port: 9092 kafka_api_tls: - name: internal enabled: true require_client_auth: true cert_file: <path-to-PEM-formatted-cert-file> key_file: <path-to-PEM-formatted-key-file> truststore_file: <path-to-PEM-formatted-CA-file> Remember to replace placeholders in brackets. kafka_api is the listener declaration. This name can have any value. advertised_kafka_api is the advertised listener. This name should match the name of a declared listener. This address is the host name clients use to connect to the broker. kafka_api_tls is the listener’s TLS configuration. This name must match the corresponding listener’s name. See also: Configure Listeners Use rpk with TLS If you’re using rpk to interact with the Kafka API using mTLS identity (for example, to manage topics or messages), pass the --tls-key, --tls-cert, and --tls-truststore flags to authenticate. To interact with the Admin API (for example, to manage users), pass the --admin-api-tls-key, --admin-api-tls-cert, and --admin-api-tls-truststore flags. rpk topic create test-topic \ --tls-key <path-to-PEM-formatted-key-file> \ --tls-cert <path-to-PEM-formatted-cert-file> \ --tls-truststore <path-to-PEM-formatted-CA-file> Replace placeholders in brackets. To check the configuration of the topic, run: rpk topic describe test-topic <tls-flags-from-above> By default, rpk connects to localhost:9092 for Kafka protocol commands. If you’re connecting to a remote broker or if you configured your local broker differently, use the -X brokers=<address:port> flag. Suggested reading TLS configuration for Redpanda and rpk Work with Schema Registry Back to top × Simple online edits For simple changes, such as fixing a typo, you can edit the content directly on GitHub. Edit on GitHub Or, open an issue to let us know about something that you want us to change. Open an issue Contribution guide For extensive content updates, or if you prefer to work locally, read our contribution guide . Was this helpful? thumb_up thumb_down group Ask in the community mail Share your feedback group_add Make a contribution Configure Authorization Configure Listeners