Streaming

Configure Authentication

Authentication verifies the identity of users and applications that connect to Redpanda clusters.

Different Redpanda APIs support different authentication methods. You can configure multiple listeners for each API, and you can configure each listener with an authentication method. Redpanda APIs support these authentication methods:

API Supported Authentication Methods

Kafka API

  • SASL

    • SASL/SCRAM

    • SASL/PLAIN

    • SASL/OAUTHBEARER (OIDC)

    • SASL/GSSAPI (Kerberos)

  • mTLS

Admin API

  • Basic authentication

  • OIDC

HTTP Proxy (PandaProxy)

  • Basic authentication

  • OIDC

Schema Registry

  • Basic authentication

  • OIDC

Users, principals, and superusers

When you configure authentication and authorization in Redpanda, it’s important to understand the distinction between users, principals, and superusers:

  • A user refers to a stored credential within Redpanda, typically a username and password stored in the internal SASL credential store. These are created using commands such as rpk security user create.

  • A principal is the authenticated identity string associated with a client session. It’s what Redpanda uses for access control, ACLs, and superuser assignment.

  • A superuser is a privileged principal who has unrestricted access to all operations in a Redpanda cluster.

Superusers can:

  • Grant or revoke permissions to other users through ACLs.

  • Access all Admin API endpoints.

  • Create, modify, or delete topics and consumer groups.

  • View and manage the cluster’s internal state.

Depending on how a client authenticates, the principal might be:

  • The SCRAM username

  • A claim from a JWT

  • A certificate DN

  • A Kerberos identity

You assign ACLs and superuser roles to principals, not users. Even if a user exists, they have no access unless its associated principal is granted permissions.

For example, with OIDC:

Configure superusers and ACL rules using the exact value from your OIDC token’s sub (subject) claim. For example, if your token’s sub claim is example@company.com, use that exact value in your configuration.

OIDC principals use the value extracted from the token’s claims according to your configured principal mapping. SASL users require the User: prefix, but OIDC principals do not use any prefix.

Enable authentication

To enable authentication in Redpanda, you must:

Create superusers

Before enabling authorization in Redpanda, which can happen implicitly when you enable authentication, you must create a superuser. Without a superuser, you can create other users, but you can’t grant them permissions to the cluster.

Enabling authorization without a superuser can result in being locked out of the cluster, as you would not have the necessary permissions to manage the cluster’s settings or users.

A superuser can either be a SCRAM user or it can be provided by external authentication mechanisms, such as OIDC. However, rpk can only communicate with the Admin API using HTTP basic authentication, which requires a SCRAM user. This means that for administrative tasks executed through rpk, you must have a SCRAM user with superuser privileges.

  1. Specify the username of a superuser.

    rpk cluster config set superusers '["superuser-username"]'

    This can be a new user or an existing user. For example, if you use the superuser named admin, then Redpanda allows the admin user to do anything, but Redpanda does not create the admin user.

  2. Create a SCRAM superuser:

    rpk security user create <superuser-username> \
  -p '<superuser-password>' \
  --mechanism=<superuser-authentication-mechanism> \
  -X admin.hosts=localhost:9644

    Replace <superuser-authentication-mechanism> with a SCRAM authentication mechanism. Valid values are SCRAM-SHA-256 or SCRAM-SHA-512.

    The Admin API defaults to localhost:9644. If you’ve configured the Admin API to use a different address/port, use the -X admin.hosts=<address:port> flag.

Now this user has full access to the cluster and can grant permissions to other users.

For information about using rpk to manage ACL users, see rpk security acl.

As a security best practice:

  • Don’t use the superuser to interact with the cluster.

  • Don’t delete the superuser (in case you need to grant permissions to new users later).

    When you create the superuser, you specify a password, which adds a level of security. If you delete the user, someone else could re-create the user with a different password.

Edit superusers

To edit a superuser, use the following command to apply the new value:

rpk cluster config edit

Enable SASL authentication

To enable authentication in your Redpanda cluster, you have the following options, depending on your requirements for SASL authentication and authorization.

You must create at least one superuser before enabling authentication. Enabling authentication without a superuser can result in being locked out of the cluster.

  • Enable SASL authentication for all Kafka listeners:

    Use this method if you haven’t already enabled authentication and you want to apply SASL authentication to all Kafka listeners. This approach does not require you to restart the cluster.

    This command implicitly enables authorization. If you want to disable authorization, you can use the kafka_enable_authorization cluster configuration property. 
    rpk cluster config set enable_sasl true

  • Explicitly enable authorization and define authentication method per listener:

    Choose this method if you require specific control over the authentication method for each Kafka listener, or if you need to enable authorization explicitly. This option requires a cluster restart.

    1. Enable authorization:

      rpk cluster config set kafka_enable_authorization true

    2. Define the authentication method for each listener. See Authentication for the Kafka API and Authentication for the HTTP APIs.

For detailed information about these and other cluster configurations, see Configure Cluster Properties.

Authentication for the Kafka API

Redpanda supports the following authentication methods for the Kafka API:

  • SASL (Simple Authentication and Security Layer)

  • mTLS (Mutual Transport Layer Security)

SASL

SASL provides a flexible and adaptable framework for implementing various authentication mechanisms. Redpanda supports these SASL mechanisms:

Enable SASL

To enable SASL authentication for the Kafka API, set the authentication_method broker property of the Kafka listeners to sasl.

If you enabled authentication with enable_sasl=true, Redpanda implicitly sets authentication_method to sasl for the Kafka listeners.

If you enabled authentication with kafka_enable_authorization=true, you must enable SASL for the Kafka listeners.

In redpanda.yaml, enter:

redpanda:
  kafka_api:
    - address: 0.0.0.0
      port: 9092
      name: sasl_listener
      authentication_method: sasl

Enable SASL with TLS encryption

SASL provides authentication, but not encryption. To provide encryption, you can enable TLS in addition to SASL. See Configure Kafka TLS Encryption.

For example, to enable SASL authentication with TLS encryption for the Kafka API, in redpanda.yaml, enter:

redpanda:
  kafka_api:
    - address: 0.0.0.0
      port: 9092
      name: sasl_tls_listener
      authentication_method: sasl
  kafka_api_tls:
    - name: sasl_tls_listener
      key_file: broker.key
      cert_file: broker.crt
      truststore_file: ca.crt
      crl_file: ca.crl # Optional
      enabled: true
      require_client_auth: false

SASL/SCRAM

SASL/SCRAM does not require sending passwords over the network, even in an encrypted form. It uses a challenge-response mechanism, ensuring that the password is not directly accessible to the server. It works with hashed passwords, providing additional security against dictionary attacks.

Enable SASL/SCRAM

SASL/SCRAM is enabled by default. To check if SASL/SCRAM is enabled:

rpk cluster config get sasl_mechanisms

You should see SCRAM in the output.

If SASL/SCRAM is not enabled, enable it by appending SCRAM to the list of SASL mechanisms:

rpk cluster config set sasl_mechanisms '["SCRAM"]'
Create SCRAM users

When you have SASL authentication enabled for your Redpanda cluster, you can create SCRAM users. Redpanda supports the following SASL/SCRAM authentication mechanisms for the Kafka API:

  • SCRAM-SHA-256

  • SCRAM-SHA-512

By default, SCRAM users don’t have any permissions in the cluster. Only superusers can grant permissions to new users through ACLs.

  1. To create the SCRAM user <my-user> with a password <change-this-password>, run rpk security user create:

    rpk security user create <my-user> \
  -p '<change-this-password>' \
  --mechanism SCRAM-SHA-256
    Enclose passwords in single quotes to avoid conflicts with special characters. Enclosing characters in single quotes preserves the literal value of each character.

  2. Use the rpk security acl create command to grant create and describe permissions to myuser in the cluster:

    rpk security acl create --allow-principal User:myuser \
  --operation create,describe \
  --cluster \
  -X user=<superuser-name> \
  -X pass='<superuser-password>' \
  -X sasl.mechanism=<superuser-authentication-mechanism>

  3. Grant the new user describe privileges for a topic called myfirsttopic:

    rpk security acl create --allow-principal User:myuser \
  --operation describe \
  --topic myfirsttopic \
  -X user=<superuser-name> \
  -X pass='<superuser-password>' \
  -X sasl.mechanism=<superuser-authentication-mechanism>
    You must grant privileges for specific topics. Even if a user has describe privileges for a cluster, it does not mean that the user is granted describe privileges for topics.

See also: User create.

Connect to Redpanda

This section provides examples of connecting to Redpanda as a SCRAM user when SASL/SCRAM authentication is enabled.

Create a topic as the myuser user by running rpk topic create:

rpk topic create myfirsttopic \
  -X user=myuser \
  -X pass='changethispassword' \
  -X sasl.mechanism=SCRAM-SHA-256

To describe the topic, run rpk topic describe:

rpk topic describe myfirsttopic \
  -X user=myuser \
  -X pass='changethispassword' \
  -X sasl.mechanism=SCRAM-SHA-256
Configure Schema Registry and HTTP Proxy to connect to Redpanda with SASL

Schema Registry and HTTP Proxy connect to Redpanda over the Kafka API.

Breaking change in Redpanda 25.2: Ephemeral credentials for HTTP Proxy are removed. If your HTTP Proxy API listeners use authentication_method: none, you must configure explicit SASL credentials (scram_username, scram_password, and sasl_mechanism) for HTTP Proxy to authenticate with the Kafka API.

This allows any HTTP API user to access Kafka using shared credentials. Redpanda Data recommends enabling HTTP Proxy authentication instead.

For details about this breaking change, see What’s new.

Schema Registry and HTTP Proxy support only the SASL/SCRAM mechanism.

  1. Create appropriate ACLs for the Schema Registry and HTTP Proxy users to define and restrict their access rights within the Redpanda cluster.

  2. Configure the listeners:

    For Schema Registry:

    schema_registry_client:
  brokers:
    - address: 127.0.0.1
      port: 9092
  scram_username: <username>
  scram_password: <password>
  sasl_mechanism: SCRAM-SHA-256

    If TLS is enabled, additional configuration is required:

    schema_registry_client:
  brokers:
    - address: 127.0.0.1
      port: 9092
  broker_tls:
    key_file: broker.key
    cert_file: broker.crt
    truststore_file: ca.crt
    crl_file: ca.crl # Optional
    enabled: true
  scram_username: <username>
  scram_password: <password>
  sasl_mechanism: SCRAM-SHA-256

    For HTTP Proxy:

    pandaproxy_client:
  brokers:
    - address: 127.0.0.1
      port: 9092
  scram_username: <username>
  scram_password: <password>
  sasl_mechanism: SCRAM-SHA-256

    When HTTP Proxy API listeners use authentication_method: none, the HTTP Proxy client uses these credentials to authenticate with the Kafka API (required starting in Redpanda 25.2). The user specified in scram_username must have appropriate permissions to access the required Kafka resources.

    If TLS is enabled for the Kafka API, additional configuration is required:

    pandaproxy_client:
  brokers:
    - address: 127.0.0.1
      port: 9092
  broker_tls:
    key_file: broker.key
    cert_file: broker.crt
    truststore_file: ca.crt
    crl_file: ca.crl # Optional
    enabled: true
  scram_username: <username>
  scram_password: <password>
  sasl_mechanism: SCRAM-SHA-256

SASL/PLAIN

You can configure Kafka clients to authenticate using either SASL/SCRAM or SASL/PLAIN with a single account using the same username and password. Unlike SASL/SCRAM, which uses a challenge response with hashed credentials, SASL/PLAIN transmits plaintext passwords. While not required, it is recommended that you use TLS for external encryption when using SASL/PLAIN authentication.

If you have existing PLAIN Kafka clients and applications, you can migrate to Redpanda without updating your application by creating local Redpanda SCRAM accounts and enabling PLAIN as an authentication mechanism.

Clusters configured with only a SASL/PLAIN mechanism are not supported.
Enable SASL/PLAIN

You must enable SASL/PLAIN explicitly by appending PLAIN to the list of SASL mechanisms:

rpk cluster config get sasl_mechanisms

- SCRAM

rpk cluster config set sasl_mechanisms '["SCRAM","PLAIN"]'

To enable SASL/PLAIN authentication for the Kafka API, set the authentication_method broker property of the Kafka listeners to sasl.

In redpanda.yaml, enter:

redpanda:
  kafka_api:
    - address: 0.0.0.0
      port: 9092
      name: sasl_plain_listener
      authentication_method: sasl

OAUTHBEARER (OIDC)

OpenID Connect (OIDC) authentication requires an enterprise license. To upgrade, contact Redpanda sales.

When you enable OIDC, Redpanda and Redpanda Console can delegate the authentication process to an external identity provider (IdP) such as Okta, Microsoft Entra ID, or on-premise Active Directory Federation Service (AD FS).

With OIDC enabled, Redpanda does not need to manage user credentials directly, but can instead rely on the trusted authentication capabilities of established IdPs.

Redpanda’s implementation of OIDC provides SASL/OAUTHBEARER support for the Kafka API, and supports standard OIDC authentication across all other HTTP APIs, including Schema Registry, HTTP Proxy, and the Admin API.

OIDC limitations

  • Redpanda requires JWT-formatted access tokens (not ID tokens) for Kafka API authentication using SASL/OAUTHBEARER. Access tokens issued by some IdPs, such as Google, are opaque and not supported.

  • The rpk CLI does not support OIDC login.

  • Redpanda requires OIDC principals to be set as superusers to access the Admin API. Granular authorization is not supported.

  • The rpk CLI does not support the SASL/OAUTHBEARER mechanism for deploying data transforms. Use SASL/SCRAM instead.

OIDC credentials flow and access token validation

Before configuring OIDC, you should understand the credentials flow, and in particular, the validation claims included in the access token, as you will need to provide them in the OIDC configuration.

Redpanda’s implementation of OIDC adheres to the client credentials flow defined in OAuth 2.0 RFC 6749, section 4.4 in which a client obtains an access token from the authorization server, and provides this access token to Redpanda, either using SASL/OAUTHBEARER for the Kafka API, or an HTTP Authorization (Bearer) header.

The access token is a bearer token. A bearer token is used for authentication and authorization in web applications and APIs, and holds user credentials, usually in the form of random strings of characters. Bearer tokens are generated based on protocols and specifications such as JWT (JSON Web Token), which has a header, payload, and signature. The signature must be verified according to the JWK. Claims inside the token and the token signature must both be validated. After validation, a configurable claim from the token payload is extracted as the principal and attached to the connection, as with any other authentication method.

Following is an example JWT header:

{
  "alg": "RS256",
  "typ": "JWT",
  "kid": "tMQzailSAdaW4nojXxES9"
}

Following is an example JWT payload:

{
  "iss": "https://dev-ltxchcls4igzho78.us.auth0.com/",
  "sub": "3JJeI4tmMC6v8mCVCSDnAGVf2vrnJ0BT@clients",
  "aud": "localhost",
  "iat": 1694430088,
  "exp": 1694516488,
  "azp": "3JJeI4tmMC6v8mCVCSDnAGVf2vrnJ0BT",
  "scope": "email2",
  "gty": "client-credentials"
}

Following are additional validation claims (JWT properties) that are included in the access token:

  • alg: The signature algorithm. The extension point in the JWT header is the signature algorithm used to sign the token, and cannot contain the value none.

  • aud: Audience. Must match the configuration specified in oidc_token_audience. Cannot contain the value none.

  • kid: Key identifier. Must match any of the public JWK listed in the jwks_uri endpoint.

  • exp: Expiration. The timestamp listed is greater than current time. Must validate within acceptable bounds of the value specified in oidc_clock_skew_tolerance. A clock skew tolerance period may be configured by an Admin to account for clock drift between Redpanda and the OIDC Identity Provider (IdP).

  • iss: Issuer. Must exactly match the issuer property of the JSON returned from the URL specified in oidc_discovery_url.

  • scope: Scope. Must include the value openid.

  • sub: Subject. This default claim identifies the principal subject. While sub is the default mapping ($.sub) in Redpanda, any claim within the JWT can be mapped to a Redpanda principal.

Enable OIDC

  1. Register a client application with your IdP.

    A client application, in this context, refers to any application or service that will authenticate against the Redpanda cluster using OIDC. This registration process involves creating a new entry in the IdP’s management console for the application, sometimes called a client. During this process, you’ll specify details about your application, such as the type of application, the callback URLs, and any other required information as per your IdP’s requirements. In an enterprise environment, OIDC integration typically requires coordination with your organization’s security team.

  2. Enable SASL authentication if it’s not already enabled.

  3. Configure ACLs for your users so they can access Redpanda resources.

  4. Enable the OAUTHBEARER SASL mechanism:

    rpk cluster config set sasl_mechanisms '["SCRAM","OAUTHBEARER"]' -X admin.hosts=localhost:9644

    Example output:

    Successfully updated configuration. New configuration version is 16.

  5. Specify the discovery URL of your identity provider (IdP). The following IdP URL uses the default value:

    rpk cluster config set oidc_discovery_url 'https://auth.prd.cloud.redpanda.com/.well-known/openid-configuration'

  6. Specify the intended audience of the token:

    rpk cluster config set oidc_token_audience 'redpanda'

  7. Specify the principal mapping, which is a JSON path that extracts a principal from any claim in the bearer token payload. The mapping rules are as follows:

    • rule = "$" segments [ mapping ]

    • segments = "." fieldname { "." fieldname }

    • mapping = "/" regex_pattern "/" replacement_pattern "/" [ case_modifier ]

    • replacement_pattern = replacement_element { replacement_element }

    • replacement_match = "$" digit

    • replacement_element = replacement_match | arbitrary_text

    • case_modifier = "L" | "U"

    For example, consider a JWT with the following claims:

    {
  "sub": "user",
  "user_info": {
    "name": "User",
    "email": "user@example.com"
  }
}

    • Default rule ($.sub): Extracts the sub claim, resulting in the principal user.

    • Extract principal from the email field ($.user_info.email/([^@]+)@.*/$1/L): This rule captures the username part of the email before the @ symbol and converts it to lowercase. The resulting principal is user.

    • Extract principal with domain validation ($.user_info.email/([^@]+)@example.com/$1/L): This rule is similar to the previous one but only applies if the email domain matches example.com. The resulting principal is user if the domain matches, otherwise, the mapping fails.

    To apply a principal mapping rule in Redpanda, use the following command:

    rpk cluster config set oidc_principal_mapping '$.sub'
rpk cluster config set oidc_principal_mapping '$.user_info.email/([^@]+)@.*/$1/L'

  8. Specify the amount of time (in seconds) to allow for when validating the expiration claim in the token:

    rpk cluster config set oidc_clock_skew_tolerance 30

  9. Enable OIDC to disconnect clients when their token expires:

    rpk cluster config set oidc_token_expire_disconnect true

  10. Specify the amount of time keys from the jwks_uri are cached:

    rpk cluster config set oidc_keys_refresh_interval 3600

GSSAPI (Kerberos)

Kerberos authentication requires an enterprise license. To upgrade, contact Redpanda sales.

To configure Kerberos authentication, use a keytab, which contains credentials for the service.

  1. Prepare the cluster:

    1. Ensure that host names are fully qualified domain names (FQDN).

    2. Ensure that each broker has a Kerberos configuration file (krb5.conf) set to use Active Directory or another corporate key distribution center (KDC). The default is at /etc/krb5.conf.

    3. Ensure that the KDC has a valid Kerberos service principal name (SPN) for each broker in the form primary/<FQDN>@<REALM>.

    4. Ensure that each broker has a keytab containing the SPN for that broker. This must be located at an identical file path on each Redpanda broker. The default is /var/lib/redpanda/redpanda.keytab.

  2. Enable SASL authentication if it’s not already enabled.

  3. Configure ACLs for your users so they can access Redpanda resources.

  4. If the keytab is not in the default location, then set its location:

    rpk cluster config set sasl_kerberos_keytab <path>

  5. If the krb5.conf file is not in the default location, then set its location:

    rpk cluster config set sasl_kerberos_config <path>

  6. Define the primary of the Kerberos SPN to be used by Redpanda with the given keytab. Default is redpanda.

    rpk cluster config set sasl_kerberos_principal <name>

  7. Set sasl_kerberos_principal_mapping. This maps Kerberos user principal names (UPNs) onto Redpanda principals used in the ACLs. For example:

    rpk cluster config set sasl_kerberos_principal_mapping '["RULE:[1:$1@$0](.*@MYDOMAIN.COM)s/@.*//","DEFAULT"]'

    By default, Redpanda matches the primary of the Kerberos UPN of the user. Each rule has the following format:

    • RULE:[n:string](regexp)s/pattern/replacement/g/c where:

      • n is an integer that indicates how many components the target principal should have.

      • If this matches, then a string is formed from string, substituting the realm of the principal for $0 and the n`’th component of the principal for `$n. (For example, if the principal is johndoe/admin@realm.com, then [2:$2$1foo] results in the string adminjohndoefoo.)

      • If this string matches regexp, then the s//[g] substitution command is run over the string.

      • g is optional. It causes the substitution to be global over the string, instead of replacing only the first match in the string.

      • c is optional. It can be either /L or /U to make the match lowercase or uppercase.

    • DEFAULT The principal name is used as the local user name. If the principal has more than one component or is not in the default realm, then the conversion fails.

      Examples of a Kerberos UPN without a host (jdoe@EXAMPLE.COM) and with a host (jdoe/host@EXAMPLE.COM):

      Translation

      jdoe@EXAMPLE.COM

      jdoe/host@EXAMPLE.COM

      [1:$1@$0]

      jdoe@EXAMPLE.COM

      Rule does not match because there are two components in the principal name jdoe/host@EXAMPLE.COM.

      [1:$1]

      jdoe

      Rule does not match because there are two components in the principal name jdoe/host@EXAMPLE.COM.

      [1:$1.foo]

      jdoe.foo

      Rule does not match because there are two components in the principal name jdoe/host@EXAMPLE.COM.

      [2:$1/$2@$0]

      Rule does not match because there is one component in the principal name jdoe@EXAMPLE.COM.

      jdoe/host@EXAMPLE.COM

      [2:$1/$2]

      Rule does not match because there is one component in the principal name jdoe@EXAMPLE.COM.

      jdoe/host

      [2:$1@$0]

      Rule does not match because there is one component in the principal name jdoe@EXAMPLE.COM.

      jdoe@EXAMPLE.COM

      [2:$1]

      Rule does not match because there is one component in the principal name jdoe@EXAMPLE.COM.

      jdoe

      DEFAULT

      jdoe

      jdoe

      The first rule that matches is used to extract a principal.

  8. Append the list of allowed SASL mechanisms that clients can use to authenticate against the Kafka API.

    To get the list of all allowed SASL mechanisms, run:

    rpk cluster config get sasl_mechanisms

    To add support for Kerberos, append the sasl_mechanisms property with the value GSSAPI:

    rpk cluster config set sasl_mechanisms '["SCRAM","GSSAPI"]'

    For Kerberos authentication, Redpanda requires that SASL/SCRAM be enabled so that rpk, Redpanda Console, and other Redpanda products can connect to the cluster. Operating with Kerberos only is not a supported configuration.

mTLS

When mTLS is enabled, both the client and the server authenticate each other using TLS certificates.

When mTLS authentication is enabled, Redpanda uses configurable rules to extract the principal from the Distinguished Name (DN) of an mTLS (X.509) certificate. It uses the principal as the identity or user name.

To enable mTLS authentication, set authentication_method broker property for a listener to mtls_identity. For example, to enable mTLS authentication for the internal Kafka API listener, in redpanda.yaml, enter:

redpanda:
  kafka_api:
    - address: 0.0.0.0
      port: 9092
      name: mtls_listener
      authentication_method: mtls_identity
  kafka_api_tls:
    - name: mtls_listener
      key_file: mtls_broker.key
      cert_file: mtls_broker.crt
      truststore_file: mtls_ca.crt
      enabled: true
      require_client_auth: true

By default, Redpanda matches the entire DN. To override the default, specify kafka_mtls_principal_mapping_rules. This is a list of rules that provide a mapping from DN to principal.

Each rule has the following format: RULE:pattern/replacement/[LU]. Where:

  • pattern is a regular expression. For example, to extract the CN field: .*CN=([^,]).*+.

  • replace is used to adjust the match. For example, to use just the first match, use: $1.

  • L makes the match lowercase (optional).

  • U makes the match uppercase (optional).

For example, with the DN: CN=www.redpanda.com,O=Redpanda,OU=Engineering,L=London,S=England,C=UK

Rule Principal

RULE:.*CN=([^,]+).*/$1/

www.redpanda.com

RULE:.*O=([^,]+).*/$1/

Redpanda

RULE:.*O=([^,]+).*/$1/L

redpanda

RULE:.*O=([^,]+),OU=([^,]+),.*,C=([^,]+)/$1-$2-$3/L

redpanda-engineering-uk

DEFAULT

CN=www.redpanda.com,O=Redpanda,OU=Engineering,L=London,S=England,C=UK

The first rule that matches is used to extract a principal.

To update the kafka_mtls_principal_mapping_rules property:

rpk cluster config set kafka_mtls_principal_mapping_rules '["DEFAULT"]'

Configure Schema Registry and HTTP Proxy to connect to Redpanda with mTLS

Schema Registry and HTTP Proxy require valid client certificates to secure the connection to Redpanda. Continuing with the previous example, where the certificate contains an identity for authentication (kafka_api listener set to mtls_identity), the following example shows how to connect Schema Registry and HTTP Proxy to Redpanda with mTLS certificate-based identity.

For example:

schema_registry_client:
  brokers:
    - address: 127.0.0.1
      port: 9092
  broker_tls:
    key_file: schema_registry.key
    cert_file: schema_registry.crt
    truststore_file: ca.crt
    enabled: true
pandaproxy_client:
  brokers:
    - address: 127.0.0.1
      port: 9092
  broker_tls:
    key_file: pandaproxy.key
    cert_file: pandaproxy.crt
    truststore_file: ca.crt
    enabled: true

Authentication for the HTTP APIs

Redpanda provides the following HTTP APIs that support authentication:

  • Admin API: Management and monitoring of the Redpanda cluster

  • Schema Registry API: Management of schemas and schema evolution

  • HTTP Proxy API: RESTful interface for Kafka clients

Permission models

Each of the HTTP APIs implements its own permission model with different levels of access control. This section lists what permissions are available to different user types, and how to enable authentication.

Admin API permissions

The Admin API primarily requires superuser privileges, with a few exceptions for read-only status endpoints.

For a complete list of all Admin API endpoints, see the Admin API reference.

Schema Registry API permissions

The Schema Registry implements a granular permission model.

Most read operations require only user-level authentication, while write operations that affect the global state or mode typically require superuser privileges.

HTTP Proxy permissions

The HTTP Proxy API impersonates the authenticated user when making Kafka API calls:

Regular users are subject to the same ACL restrictions as they would be when using the Kafka API directly.

Enable authentication

Redpanda supports authentication for the HTTP APIs using either basic authentication or OIDC (OpenID Connect).

Prerequisites

Before enabling authentication for the HTTP APIs, you must enable SASL authentication for the Kafka API. This creates the credential store that HTTP authentication will use.

Basic authentication

Redpanda Data recommends that you use TLS when enabling HTTP Basic Auth.

Basic authentication provides a method for securing HTTP endpoints. With basic authentication enabled, HTTP user agents, such as web browsers, must provide a username and password when making a request.

To add users to the Redpanda credential store that HTTP basic authentication uses, create users with rpk security user create.

The HTTP Proxy API and the Schema Registry API use the same credential store as the Kafka API, so you can use the same credentials for all three APIs. The Admin API uses the same credential store as the Kafka API, but it requires superuser credentials to access it.

To enable basic authentication for the Admin API:

  1. create a SCRAM superuser so that you can use rpk to create ACLs.

    rpk supports only basic authentication for the Admin API.

  2. Enable authentication for the Admin API:

    rpk cluster config set admin_api_require_auth true

  3. Enable basic authentication:

    rpk cluster config set http_authentication '["BASIC"]'
Valid values for the cluster configuration property http_authentication (cluster-wide) are BASIC and OIDC. The value BASIC here is different from the per-listener setting http_basic, which enables authentication on a listener using the broker property authentication_method (see authentication_method for the Schema Registry listener and authentication_method for the HTTP Proxy listener).

To enable basic authentication for specific listeners, set authentication_method broker property to http_basic. For example, in redpanda.yaml, enter:

pandaproxy:
  pandaproxy_api:
    address: "localhost"
    port: 8082
    authentication_method: http_basic
schema_registry:
  schema_registry_api:
    address: "localhost"
    port: 8081
    authentication_method: http_basic
Connect to the HTTP API

To access the internal listener:

curl http://localhost:8082/topics -u <username>:<password> -sS

If TLS is enabled, specify the HTTPS protocol and pass the path to the ca.crt file:

curl https://localhost:8082/topics --cacert <path-to-ca>/ca.crt -u <username>:<password> -sS
If the broker’s certificate is signed by a well-known, trusted CA, and you’re confident about the integrity of your system’s CA trust store, you don’t need the --cacert flag.

For all available endpoints, see the HTTP Proxy API reference.

Connect to the Schema Registry API

To access the internal listener:

curl http://localhost:8081/subjects -u <username>:<password> -sS

If TLS is enabled, specify the HTTPS protocol and pass the path to the ca.crt file:

curl https://localhost:8081/subjects --cacert <path-to-ca>/ca.crt -u <username>:<password> -sS
If the broker’s certificate is signed by a well-known, trusted CA, and you’re confident about the integrity of your system’s CA trust store, you don’t need the --cacert flag.

For all available endpoints, see the Schema Registry API.

OIDC

You can configure the HTTP APIs to authenticate users with the OIDC bearer token. By using OIDC, you can centralize credentials and provide a password-free SSO experience.

See Enable OIDC to configure the required OIDC cluster configuration properties before enabling OIDC for the HTTP APIs. You can configure OIDC without enabling it for the Kafka API.

If you enable OIDC authentication for the Admin API, you must also create a SCRAM superuser so that you can use rpk to create ACLs. rpk supports only basic authentication for the Admin API. See Authentication for the HTTP APIs.

To enable OIDC for the HTTP API listeners as well as basic authentication, include OIDC in the http_authentication cluster property list:

Valid values for the cluster configuration property http_authentication (cluster-wide) are BASIC and OIDC. The value BASIC here is different from the per-listener setting http_basic, which enables authentication on a listener using the broker property authentication_method (see authentication_method for the Schema Registry listener and authentication_method for the HTTP Proxy listener). 
rpk cluster config set http_authentication '["BASIC","OIDC"]'

To enable OIDC for HTTP API listeners, set authentication_method to http_basic to require authentication on those listeners. For example, in redpanda.yaml, enter:

pandaproxy:
  pandaproxy_api:
    address: "localhost"
    port: 8082
    authentication_method: http_basic  # Requires authentication (Basic or OIDC)
schema_registry:
  schema_registry_api:
    address: "localhost"
    port: 8081
    authentication_method: http_basic  # Requires authentication (Basic or OIDC)
The authentication_method broker property controls whether a listener requires authentication (http_basic) or allows anonymous access (none). The actual choice between Basic authentication and OIDC authentication is determined by:

  • What authentication methods are enabled in the http_authentication cluster property

  • What type of Authorization header the client sends (Basic for Basic auth, Bearer for OIDC)

Connect to the HTTP API

To access the internal listener:

curl http://localhost:8082/topics -H "Authorization: Bearer <bearer-token>" -sS

If TLS is enabled, specify the HTTPS protocol and pass the path to the ca.crt file:

curl https://localhost:8082/topics --cacert <path-to-ca>/ca.crt -H "Authorization: Bearer <bearer-token>" -sS
If the broker’s certificate is signed by a well-known, trusted CA, and you’re confident about the integrity of your system’s CA trust store, you don’t need the --cacert flag.

For all available endpoints, see the HTTP Proxy API reference.

Connect to the Schema Registry API

To access the internal listener:

curl http://localhost:8081/subjects -H "Authorization: Bearer <bearer-token>" -sS

If TLS is enabled, specify the HTTPS protocol and pass the path to the ca.crt file:

curl https://localhost:8081/subjects --cacert <path-to-ca>/ca.crt -H "Authorization: Bearer <bearer-token>" -sS
If the broker’s certificate is signed by a well-known, trusted CA, and you’re confident about the integrity of your system’s CA trust store, you don’t need the --cacert flag.

For all available endpoints, see the Schema Registry API.

Disable authentication

To disable authentication for a listener, set authentication_method broker property to none:

Breaking change in Redpanda 25.2: Ephemeral credentials for HTTP Proxy are removed. If your HTTP Proxy API listeners use authentication_method: none, you must configure explicit SASL credentials (scram_username, scram_password, and sasl_mechanism) for HTTP Proxy to authenticate with the Kafka API.

This allows any HTTP API user to access Kafka using shared credentials. Redpanda Data recommends enabling HTTP Proxy authentication instead.

For details about this breaking change, see What’s new.

pandaproxy:
  pandaproxy_api:
    address: "localhost"
    port: 8082
    authentication_method: none
schema_registry:
  schema_registry_api:
    address: "localhost"
    port: 8081
    authentication_method: none

If authorization is disabled, connections to this listener use the anonymous user.

To disable authentication on the Kafka API, run:

rpk cluster config set enable_sasl false

Or, set the authentication_method of the Kafka listeners to none:

redpanda:
  kafka_api:
    - address: 0.0.0.0
      port: 9092
      name: sasl_listener
      authentication_method: none

Next steps

Suggested reading

Suggested labs

Search all labs