Manage Secrets

Learn how to manage secrets in Redpanda Connect, and how to add them to your data pipelines without exposing them.

Secrets are stored in the secret management solution of your Cloud provider and are retrieved when you run a pipeline configuration that references them.

Prerequisites

  • A running BYOC, Dedicated or Serverless cluster

  • On BYOC clusters only, check that secrets management is enabled:

    1. Log in to Redpanda Cloud.

    2. Go to the Connect page.

    3. Select the Redpanda Connect tab and make sure you can see a Secrets tab.

    If you cannot see a Secrets tab, contact Redpanda Support

Manage secrets

You can manage secrets from the Cloud UI or Data Plane API.

Create a secret

You can create a secret and reference it in multiple data pipelines on the same cluster.

  • Cloud UI (BYOC and Dedicated)

  • Cloud UI (Serverless)

  • Data Plane API

  1. Log in to Redpanda Cloud.

  2. Go to the Connect page.

  3. Select the Redpanda Connect tab and then the Secrets tab.

  4. Click Create secret.

  5. In Secret name, enter a name for the secret. You cannot rename the secret once it is created.

  6. In Secret value, enter the secret you need to add.

  7. Click Create secret.

    The secret details are listed in the Secrets tab ready to add to your data pipelines.

  1. Log in to Redpanda Cloud.

  2. Go to the Connect page.

  3. Select the Secrets tab and click Create secret.

  4. In Secret name, enter a name for the secret. You cannot rename the secret once it is created.

  5. In Secret value, enter the secret you need to add.

  6. Click Create secret.

    The secret details are listed in the Secrets tab ready to add to your data pipelines.

You must use a Base64-encoded secret.

  1. Authenticate and get the base URL for the Data Plane API.

  2. Make a request to POST /v1alpha2/secrets.

    curl -X POST "https://<dataplane-api-url>/v1alpha2/secrets" \
 -H 'accept: application/json'\
 -H 'authorization: Bearer <token>'\
 -H 'content-type: application/json' \
 -d '{"id":"<secret-name>","scopes":["SCOPE_REDPANDA_CONNECT"],"secret_data":"<secret-value>"}'

    You must include the following values:

    • <dataplane-api-url>: The base URL for the Data Plane API.

    • <token>: The API key you generated during authentication.

    • <secret-name>: The ID or name of the secret you want to add. Use only the following characters: ^[A-Z][A-Z0-9_]*$.

    • <secret-value>: The Base64-encoded secret.

    • This scope: "SCOPE_REDPANDA_CONNECT".

    The response returns the name of the secret and the scope "SCOPE_REDPANDA_CONNECT".

You can now add the secret to your data pipeline.

Update a secret

You can only update the secret value, not its name.

Changes to secret values do not take effect until a pipeline is restarted.

  • Cloud UI (BYOC and Dedicated)

  • Cloud UI (Serverless)

  • Data Plane API

  1. Log in to Redpanda Cloud.

  2. Go to the Connect page.

  3. Select the Redpanda Connect tab and then the Secrets tab.

  4. Find the secret you want to update and click the edit icon.

  5. Enter the new secret value and click Update Secret.

  6. Start and stop any pipelines that reference the secret.

  1. Log in to Redpanda Cloud.

  2. Go to the Connect page.

  3. Select the Secrets tab.

  4. Find the secret you want to update and click the edit icon.

  5. Enter the new secret value and click Update Secret.

  6. Start and stop any pipelines that reference the secret.

You must use a Base64-encoded secret.

  1. Authenticate and get the base URL for the Data Plane API.

  2. Make a request to PUT /v1alpha2/secrets/{id}.

    curl -X PUT "https://<dataplane-api-url>/v1alpha2/secrets/<secret-name>" \
 -H 'accept: application/json'\
 -H 'authorization: Bearer <token>'\
 -H 'content-type: application/json' \
 -d '{"scopes":["SCOPE_REDPANDA_CONNECT"],"secret_data":"<secret-value>"}'

    You must include the following values:

    • <dataplane-api-url>: The base URL for the Data Plane API.

    • <secret-name>: The name of the secret you want to update.

    • <token>: The API key you generated during authentication.

    • This scope: "SCOPE_REDPANDA_CONNECT".

    • <secret-value>: Your new Base64-encoded secret.

    The response returns the name of the secret and the scope "SCOPE_REDPANDA_CONNECT".

Delete a secret

Before you delete a secret, make sure that you remove references to it from your data pipelines.

Changes do not affect pipelines that are already running.

  • Cloud UI (BYOC and Dedicated)

  • Cloud UI (Serverless)

  • Data Plane API

  1. Log in to Redpanda Cloud.

  2. Go to the Connect page.

  3. Select the Redpanda Connect tab and then the Secrets tab.

  4. Find the secret you want to remove and click the delete icon.

  5. Confirm your deletion.

  1. Log in to Redpanda Cloud.

  2. Go to the Connect page.

  3. Select the Secrets tab.

  4. Find the secret you want to remove and click the delete icon.

  5. Confirm your deletion.

  1. Authenticate and get the base URL for the Data Plane API.

  2. Make a request to DELETE /v1alpha2/secrets/{id}.

    curl -X DELETE "https://<dataplane-api-url>/v1alpha2/secrets/<secret-name>" \
 -H 'accept: application/json'\
 -H 'authorization: Bearer <token>'\

    You must include the following values:

    • <dataplane-api-url>: The base URL for the Data Plane API.

    • <secret-name>: The name of the secret you want to delete.

    • <token>: The API key you generated during authentication.

Add a secret to a data pipeline

You can add a secret to any pipeline in your cluster using the notation ${secrets.SECRET_NAME}. In the Cloud UI, you can copy the notation from the Secrets tab.

For example:

    sasl:
      - mechanism: SCRAM-SHA-256
        username: "user"
        password: "${secrets.PASSWORD}"