Authentication

Redpanda Cloud ensures the highest level of authentication for both users and services.

User authentication

Redpanda provides user authentication to your Redpanda organization through email or single sign-on.

Email

Redpanda Cloud can authenticate users with emails and passwords. Passwords are hashed (a one-way function that makes the original value unrecoverable, and effectively encrypted) and salted at rest using bcrypt.

Single sign-on

To unlock this feature for your account, contact Redpanda support.

Redpanda Cloud can authenticate users with single sign-on (SSO) to an OIDC-based identity provider (IdP). Redpanda integrates with any OIDC-compliant IdP that supports discovery, including Okta, Auth0, Microsoft Entra, and Active Directory Federation Services (AD-FS). After SSO is enabled for an organization, new users in that organization can authenticate with SSO.

Integrate IdP

You must integrate your IdP with Redpanda Cloud to use SSO. On the Users page, users with admin permission see a Single sign-on tab and can add connections for up to two different IdPs. Enter the client ID, client secret, and discovery URI for the IdP. (See your IdP documentation for these values. The discovery URI may be called something different, like the well known URL or the issuer_url.)

By default, the connection is added in a disabled state. Edit the connection to enable it. You can also choose to enable auto-enroll in the connection, which provides new users signing in from that IdP access to your Redpanda organization. When you enable auto-enroll, you select to assign a read, write, or admin role to users who log in with that IdP.

Set up is different for most IdPs. For example, for Okta, follow the Okta documentation to create an application within Okta for Redpanda. The Redpanda callback location (that is, the redirect location where Okta sends the user) is the following:

https://auth.prd.cloud.redpanda.com/login/callback

Okta provides the following fields required for SSO configuration on the Users page: clientId, clientSecret, and discoveryUrl. The discovery URL for Okta generally looks like the following (where an_id could be “default”):

https://<orgname>.okta.com/oauth2/<an_id>/.well-known/openid-configuration
Deleting an SSO connection also deletes all users attached to it.

Service authentication

Each Redpanda Cloud data plane runs its own dedicated agent, which authenticates and connects against the control plane over a single TLS 1.2 encrypted TCP connection.

Redpanda Cloud enables SASL/SCRAM authentication over TLS 1.2 as well as mTLS to authenticate Kafka clients connecting to Redpanda clusters over the TCP endpoint or listener.

When connecting through Redpanda’s HTTP Proxy, authentication is done through an HTTP Basic Authentication header encrypted over TLS 1.2.

The following features use AWS and GCP IAM Policies to generate dynamic and short-lived credentials to interact with cloud provider APIs:

  • Data plane agent

  • Tiered Storage

  • Redpanda Console

  • Managed connectors

AWS and GCP IAM policies have constrained permissions so that each service can only access or manage its own data plane-scoped resources, following the principle of least privilege.

Enable mTLS Authentication for Kafka API

Redpanda Cloud supports mTLS authentication for the Kafka API.

Requirements

  • A service account in your Redpanda organization. You must have administrative privileges to create a service account.

  • Use the Cloud API to enable mTLS. You use the service account to obtain an access token. You then make a request to the API to update an existing cluster to use mTLS, passing the access token in the authorization header.

Use the Cloud API to enable mTLS

  1. Create a service account in your organization, if you have not already done so. Go to the Clients page in the Redpanda Cloud UI and click Add client to create a service account. Enter a name and description.

  2. Retrieve the client ID and secret by clicking Copy ID and Copy Secret.

  3. Obtain an access token by making a POST request to https://auth.prd.cloud.redpanda.com/oauth/token with the ID and secret in the request body.

  4. Make a PATCH /v1beta2/clusters/{cluster.id} request to enable mTLS for the Kafka API on a cluster.

The following code block shows a request for an access token, followed by a request to enable mTLS:

AUTH_TOKEN=`curl -s --request POST \
    --url 'https://auth.prd.cloud.redpanda.com/oauth/token' \
    --header 'content-type: application/x-www-form-urlencoded' \
    --data grant_type=client_credentials \
    --data client_id=<client-id> \
    --data client_secret=<client-secret> \
    --data audience=cloudv2-production.redpanda.cloud | jq .access_token | sed 's/"//g'`

CLUSTER_PATCH_BODY=`cat << EOF
{
  "kafka_api": {
     "mtls": {
        "enabled": true,
        "ca_certificates_pem": ["<ca-certificate-pem>"],
        "principal_mapping_rules": ["<principal-mapping-rule>"]
     }
  }
}
EOF`
curl -v -X PATCH \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $AUTH_TOKEN" \
-d "$CLUSTER_PATCH_BODY" https://api.redpanda.com/v1beta2/clusters/<cluster-id>`

Make sure to replace the following variables:

  • <client-id>: Client ID.

  • <client-secret>: Client secret.

  • <cluster-id>: ID of Redpanda cluster.

  • <ca-certificate-pem>: A trusted Kafka client CA certificate in PEM format. The ca_certificates_pem field accepts a list of certificates.

  • <principal-mapping-rule>: Configurable rule for mapping the Distinguished Name of Kafka client certificates to Kafka principals.

    For example, the mapping rule RULE:.*CN=([^,]+).*/\\$1/ maps the following certificate subject to a principal named test:

    Subject: C=US, ST=IL, L=Chicago, O=redpanda, OU=cloud, CN=test, emailAddress=test123@redpanda.com

    See Configure Authentication for more details on principal mapping rules. The principal_mapping_rules field accepts a list of rules.

Verify that mTLS is enabled

To verify that mTLS is enabled for the Kafka API, run the following rpk command, without providing a security certificate or key:

rpk cluster info --tls-enabled

You should get the following error:

unable to request metadata: remote error: tls: certificate required

Consume and produce with mTLS enabled

When you consume, produce to, or manage topics using rpk, you must provide a client certificate and private key. You may use the --tls-cert and --tls-key options, or environment variables with rpk.

rpk topic create test-topic --tls-enabled --tls-cert=/path/to/tls.crt --tls-key=/path/to/tls.key

Suggested reading