Skip to main content
Version: 22.2

Configuring Authentication

Redpanda supports multiple forms of authentication including SASL/SCRAM, mTLS with principal mapping, and mixed-mode with multiple listeners. You can configure multiple listeners with redpanda.yaml, and with each listener, you can configure authentication_method and optionally TLS or mTLS. The guidelines put in place by your organization determine the type of authentication that you use.

See also:

Enable authentication

To enable authentication, set kafka_enable_authorization to true, and specify at least one value for the superusers property. This user is used to bootstrap permissions for other users in the cluster. See Cluster configuration properties.

To update the kafka_enable_authorization property, run:

rpk cluster config set kafka_enable_authorization true

You can edit the superusers similarly. Redpanda recommends that you use the edit command to apply the new value:

rpk cluster config edit

To specify superusers using the set command, run:

rpk cluster config set superusers ['admin']

Create superusers

When you configure authentication, you include one or more superusers in the Redpanda configuration file. This user has ALL permissions on the cluster and grants permissions to new users. Without a superuser, you can create other users, but you can't grant them permissions to the cluster.

You can specify the name of the superuser. For example, if you used superuser admin, Redpanda allows the admin user to do anything, but Redpanda does not create the admin user. To create this superuser, run:

rpk acl user create <super_user_username> -p <super_user_password> 

You can specify another existing user or a user that does not exist yet. Adding the user name in the configuration file does not create the user, but when you create a user with the user name specified in the configuration file, that user has full access to the cluster. You can then use that superuser to grant permissions to other users.

The Admin API defaults to localhost:9644. If you've configured the Admin API to use a different address/port, use the --api-urls <address:port> flag.

Creating ACLs for users uses the Kafka protocol (default of localhost:9092). Because no ACLs exist at the start, you need a superuser to bypass the requirement of needing ACLs to create ACLs.

To create users and set passwords, run:

rpk acl user create admin \
-p <password> \
--api-urls localhost:9644

For information about using rpk to manage ACL users, see rpk acl.

As a security best practice, you don't want to use the superuser to interact with the cluster, but you also don't want to delete the superuser (in case you need to grant permissions to new users later on). In addition, when you create the superuser, you specify a password for the user, which adds a level of security to the superuser. If you delete the user, someone else might re-create the user with a different password.

SASL/SCRAM

Simple Authentication Security Layer (SASL) is an authentication framework that lets the user choose the authentication mechanism. Redpanda supports the Salted Challenge Response Authentication Mechanism (SCRAM) authentication method.

note

SASL authentication is only available for the Kafka API.

SASL provides authentication, but not encryption. You can, however, configure TLS to only handle encryption, and use SASL for authentication. This is useful if you require flexibility in the authorization mechanisms that you use.

SCRAM provides strong encryption for user names and passwords by default and does not require an external data store for user information. Redpanda supports the following SASL mechanisms:

  • SCRAM-SHA-256
  • SCRAM-SHA-512

When you run a command with SASL authentication, you must include the mechanism with the following flag:

--sasl-mechanism <mechanism>

For example, to use the SCRAM-SHA-256 mechanism, run:

rpk topic create littlefoot \
--user panda \
--password pandaPassword \
--sasl-mechanism SCRAM-SHA-256

Configure SASL authentication

To configure SASL authentication for the Kafka API, set authentication_method of the listener to sasl.

In redpanda.yaml, enter:

redpanda:
kafka_api:
- address: 0.0.0.0
port: 9092
name: sasl_listener
authentication_method: sasl

Enable SASL with TLS encryption

To enable SASL authentication with TLS encryption for the Kafka API, in redpanda.yaml, enter:

redpanda:
kafka_api:
- address: 0.0.0.0
port: 9092
name: sasl_tls_listener
authentication_method: sasl
kafka_api_tls:
- name: sasl_tls_listener
key_file: server.key
cert_file: server.crt
truststore_file: ca.crt
enabled: true
require_client_auth: false

Configure SASL for HTTP Proxy and Schema Registry

The HTTP Proxy and Schema Registry connect to Redpanda over the Kafka API, and must be configured with a username and password.

For schema registry, the configuration node is schema_registry_client:

schema_registry_client:
brokers:
- address: 127.0.0.1
port: 9092
scram_username: panda
scram_password: pandaPassword
sasl_mechanism: SCRAM-SHA-256

If TLS is in use, additional configuration is required:

schema_registry_client:
brokers:
- address: 127.0.0.1
port: 9092
broker_tls:
key_file: server.key
cert_file: server.crt
truststore_file: ca.crt
enabled: true
scram_username: panda
scram_password: pandaPassword
sasl_mechanism: SCRAM-SHA-256

HTTP Proxy has a similar configuration, but the configuration node is pandaproxy_client:

pandaproxy_client:
brokers:
- address: 127.0.0.1
port: 9092
broker_tls:
key_file: server.key
cert_file: server.crt
truststore_file: ca.crt
enabled: true
scram_username: panda
scram_password: pandaPassword
sasl_mechanism: SCRAM-SHA-256

Connect to Redpanda

You can use the newly-created user to interact with Redpanda with rpk:

rpk topic describe test-topic \
--user admin \
--password <password> \
--sasl-mechanism SCRAM-SHA-256 \
--brokers localhost:9092
SUMMARY
=======
NAME test-topic
PARTITIONS 1
REPLICAS 1

CONFIGS
=======
KEY VALUE SOURCE
cleanup.policy delete DYNAMIC_TOPIC_CONFIG
compression.type producer DEFAULT_CONFIG
message.timestamp.type CreateTime DEFAULT_CONFIG
...

See also: Configuring Redpanda SASL on Kubernetes

Configure mTLS with authentication

For mTLS authentication, Redpanda uses configurable rules to extract the principal from the Distinguished Name (DN) of an mTLS (X.509) certificate. It uses the principal as the identity or user name.

To enable mTLS authentication, set the authentication_method of the listener to mtls_identity.

In redpanda.yaml, enter:

redpanda:
kafka_api:
- address: 0.0.0.0
port: 9092
name: mtls_listener
authentication_method: mtls_identity
kafka_api_tls:
- name: mtls_listener
key_file: mtls_server.key
cert_file: mtls_server.crt
truststore_file: mtls_ca.crt
enabled: true
require_client_auth: true

By default, Redpanda matches the entire DN. To override the default, specify kafka_mtls_principal_mapping_rules. This is a list of rules that provide a mapping from DN to principal. Each rule has the following format: RULE:pattern/replacement/[LU]. Where:

  • pattern is a regular expression. For example, to extract the CN field: .*CN=([^,]).*
  • replace is used to adjust the match. For example, to use just the first match, use: $1.
  • L makes the match lowercase (optional)
  • U makes the match uppercase (optional)

For example, assume a DN: CN=www.redpanda.com,O=Redpanda,OU=Engineering,L=London,S=England,C=UK

RulePrincipal
RULE:.*CN=([^,]).*/$1/www.redpanda.com
RULE:.*O=([^,]).*/$1/Redpanda
RULE:.*O=([^,]).*/$1/Lredpanda
RULE:.*O=([^,]),OU=([^,]),.*,C=([^,])/$1-$2-$3/Lredpanda-engineering-uk
DEFAULTCN=www.redpanda.com,O=Redpanda,OU=Engineering,L=London,S=England,C=UK

The first rule that matches is used to extract a principal.

To update the kafka_mtls_principal_mapping_rules property, run:

rpk cluster config set kafka_mtls_principal_mapping_rules '["DEFAULT"]'

Configure mTLS for HTTP Proxy and Schema Registry

The HTTP Proxy and Schema Registry require valid client certificates.

In redpanda.yaml, enter:

schema_registry_client:
brokers:
- address: 127.0.0.1
port: 9092
broker_tls:
key_file: schema_registry.key
cert_file: schema_registry.crt
truststore_file: ca.crt
enabled: true
pandaproxy_client:
brokers:
- address: 127.0.0.1
port: 9092
broker_tls:
key_file: pandaproxy.key
cert_file: pandaproxy.crt
truststore_file: ca.crt
enabled: true

See also: Configuring Redpanda mTLS on Kubernetes

Disable authentication

To disable authentication for a listener, set authentication_method to none. If authorization is enabled, connections to this listener use the anonymous user.