Different components of Redpanda support different authentication methods. You can configure multiple listeners with
redpanda.yaml, and with each listener, you can configure the
authentication_method and optionally TLS or mTLS. The guidelines put in place by your organization determine the type of authentication that you use.
|Passwords are in single quotes to avoid conflicts with special characters. Ensure that all values entered are properly enclosed in quotes and escaped as necessary.|
To enable authentication, set
true, and specify at least one value for the
superusers property. This superuser is used to bootstrap permissions for other users in the cluster. See Cluster configuration properties.
To update the
kafka_enable_authorization property, run:
rpk cluster config set kafka_enable_authorization true
To specify a superuser, run:
rpk cluster config set superusers ['admin']
To edit a superuser, use the edit command to apply the new value:
rpk cluster config edit
When you configure authentication, you include one or more superusers in the Redpanda configuration file. This user has ALL permissions on the cluster and grants permissions to new users. (Without a superuser, you can create other users, but you can’t grant them permissions to the cluster.)
Specify the name of the superuser. This can be a new user or an existing user. For example, if you use the superuser named
admin, then Redpanda allows the
admin user to do anything, but Redpanda does not create the
admin user. To create this superuser, run:
rpk acl user create <superuser_username> -p '<superuser_password>'
admin user has full access to the cluster and can grant permissions to other users.
The Admin API defaults to
localhost:9644. If you’ve configured the Admin API to use a different address/port, use the
--api-urls <address:port> flag.
Creating ACLs for users uses the Kafka protocol (default of
localhost:9092). Because no ACLs exist at the start, you need a superuser to bypass the requirement of needing ACLs to create ACLs.
To create users and set passwords, run:
rpk acl user create admin \ -p '<password>' \ --api-urls localhost:9644
For information about using
rpk to manage ACL users, see rpk acl.
|As a security best practice, don’t use the superuser to interact with the cluster, and don’t delete the superuser (in case you need to grant permissions to new users later). In addition, when you create the superuser, you specify a password, which adds a level of security. If you delete the user, someone else could re-create the user with a different password.|
Simple Authentication Security Layer (SASL) is an authentication framework that lets the user choose the authentication mechanism. Redpanda supports the Salted Challenge Response Authentication Mechanism (SCRAM) authentication method.
|SASL authentication is only available for the Kafka API.|
SASL provides authentication, but not encryption. You can, however, configure TLS to only handle encryption, and use SASL for authentication. This is useful if you require flexibility in your authorization mechanisms.
SCRAM provides strong encryption for user names and passwords by default and does not require an external data store for user information. Redpanda supports the following SASL mechanisms:
When you run a command with SASL authentication, you must include the mechanism with the following flag:
For example, to use the
SCRAM-SHA-256 mechanism, run:
rpk topic create littlefoot \ --user <username> \ --password <password> \ --sasl-mechanism SCRAM-SHA-256
To configure SASL authentication for the Kafka API, set
authentication_method of the listener to
redpanda: kafka_api: - address: 0.0.0.0 port: 9092 name: sasl_listener authentication_method: sasl
To enable SASL authentication with TLS encryption for the Kafka API, in
redpanda: kafka_api: - address: 0.0.0.0 port: 9092 name: sasl_tls_listener authentication_method: sasl kafka_api_tls: - name: sasl_tls_listener key_file: server.key cert_file: server.crt truststore_file: ca.crt enabled: true require_client_auth: false
Schema Registry and HTTP Proxy connect to Redpanda over the Kafka API. For the Kafka username and password, Redpanda uses ephemeral credentials internal to the cluster. Ephemeral credentials are regular SCRAM credentials, but they’re only stored in memory and are lost when a node restarts. When the Schema Registry or HTTP Proxy start up, they broadcast an ephemeral credential to other nodes over the internal RPC. If authentication fails to a particular broker, new ephemeral credentials are sent to that broker, and the service reconnects.
You can override the ephemeral credentials and manually configure Schema Registry and HTTP Proxy to connect to Redpanda with SASL.
The Schema Registry configuration node is
schema_registry_client: brokers: - address: 127.0.0.1 port: 9092 scram_username: <username> scram_password: <password> sasl_mechanism: SCRAM-SHA-256
If TLS is in use, additional configuration is required:
schema_registry_client: brokers: - address: 127.0.0.1 port: 9092 broker_tls: key_file: server.key cert_file: server.crt truststore_file: ca.crt enabled: true scram_username: <username> scram_password: <password> sasl_mechanism: SCRAM-SHA-256
HTTP Proxy has a similar configuration, but the configuration node is
pandaproxy_client: brokers: - address: 127.0.0.1 port: 9092 broker_tls: key_file: server.key cert_file: server.crt truststore_file: ca.crt enabled: true scram_username: <username> scram_password: <password> sasl_mechanism: SCRAM-SHA-256
You can use the newly-created user to interact with Redpanda with
rpk topic describe test-topic \ --user admin \ --password <password> \ --sasl-mechanism SCRAM-SHA-256 \ --brokers localhost:9092
SUMMARY ======= NAME test-topic PARTITIONS 1 REPLICAS 1 CONFIGS ======= KEY VALUE SOURCE cleanup.policy delete DYNAMIC_TOPIC_CONFIG compression.type producer DEFAULT_CONFIG message.timestamp.type CreateTime DEFAULT_CONFIG ...
|Basic authentication is supported on the Admin API, Schema Registry, and HTTP Proxy.|
To configure basic authentication on the Admin API, set
Administrators create users with
rpk acl user create. This adds users to the Redpanda credential store that HTTP basic authentication uses.
You can enable basic authentication to use Kafka API username/password credentials to authenticate to HTTP Proxy and Schema Registry. This requires that SASL is turned on for Kafka API endpoints.
HTTP Proxy: Access to the Kafka API impersonates the user whose credentials were used to authenticate to HTTP Proxy, and the user is subject to authorization restrictions by Redpanda ACLs. To support this design, Redpanda passes the username/password in memory to a SASL-enabled Kafka client.
Schema Registry: Authorization is "all or nothing": if the user presents a valid user account, then they have full read/write access.
To configure basic authentication, set
pandaproxy: pandaproxy_api: - address: "localhost" port: 8082 authentication_method: http_basic schema_registry: schema_registry_api: address: "localhost" port: 8081 authentication_method: http_basic
Then to use basic authentication:
rpk acl user create foo --password 'bar' # Creates SASL user "foo" for the Kafka API curl -u "foo:bar" "http://localhost:8082/topics" # A request to the HTTP Proxy with user foo. Don't forget the colon! curl -u "foo:bar" "http://localhost:8081/subjects" # A request to the Schema Registry with user foo. Don't forget the colon!
For mTLS authentication, Redpanda uses configurable rules to extract the principal from the Distinguished Name (DN) of an mTLS (X.509) certificate. It uses the principal as the identity or user name.
To enable mTLS authentication, set
redpanda: kafka_api: - address: 0.0.0.0 port: 9092 name: mtls_listener authentication_method: mtls_identity kafka_api_tls: - name: mtls_listener key_file: mtls_server.key cert_file: mtls_server.crt truststore_file: mtls_ca.crt enabled: true require_client_auth: true
By default, Redpanda matches the entire DN. To override the default, specify
kafka_mtls_principal_mapping_rules. This is a list of rules that provide a mapping from DN to principal.
Each rule has the following format:
patternis a regular expression. For example, to extract the CN field:
replaceis used to adjust the match. For example, to use just the first match, use:
Lmakes the match lowercase (optional).
Umakes the match uppercase (optional).
For example, with the DN:
The first rule that matches is used to extract a principal.
To update the
kafka_mtls_principal_mapping_rules property, run:
rpk cluster config set kafka_mtls_principal_mapping_rules '["DEFAULT"]'
Schema Registry and HTTP Proxy require valid client certificates to secure the connection to Redpanda. Continuing with the previous example, where the certificate contains an identity for authentication (
kafka_api listener set to
mtls_identity), the following example shows how to connect Schema Registry and HTTP Proxy to Redpanda with mTLS certificate-based identity.
schema_registry_client: brokers: - address: 127.0.0.1 port: 9092 broker_tls: key_file: schema_registry.key cert_file: schema_registry.crt truststore_file: ca.crt enabled: true pandaproxy_client: brokers: - address: 127.0.0.1 port: 9092 broker_tls: key_file: pandaproxy.key cert_file: pandaproxy.crt truststore_file: ca.crt enabled: true
To disable authentication for a listener, set
pandaproxy: pandaproxy_api: - address: "localhost" port: 8082 authentication_method: none schema_registry: schema_registry_api: address: "localhost" port: 8081 authentication_method: none
If authorization is enabled, connections to this listener use the anonymous user.
To disable authentication on the Kafka API, disable
kafka_enable_authorization and set
none for all listeners.
For example, run
rpk cluster config set kafka_enable_authorization false, and set the following:
redpanda: kafka_api: - address: 0.0.0.0 port: 9092 name: sasl_listener authentication_method: none