Connect

Authenticate to Amazon Aurora using IAM roles

Use IAM roles to authenticate self-hosted Redpanda Connect pipelines to Amazon Aurora on AWS without storing static credentials.

Self-hosted deployments only

This guide applies to self-hosted Redpanda Connect deployments. Redpanda Cloud pipelines in Bring Your Own Cloud (BYOC) and Dedicated clusters do not currently support IAM Roles for Service Accounts (IRSA), and cross-account sts:AssumeRole from the Redpanda-managed pipeline role is not available because you cannot grant additional permissions to that role. The Redpanda Connect binary supports both, but the Cloud platform does not currently make pod identity tokens available to pipelines. To authenticate Cloud pipelines to AWS services, store static credentials as secrets and reference them in your pipeline configuration. See Manage Secrets in Redpanda Cloud.

Authentication uses a two-hop role chain. The Redpanda Connect process assumes an IAM role through its runtime environment, such as IRSA on Amazon EKS or an EC2 instance profile. That role then assumes a database-specific role in your Aurora account, which generates a short-lived RDS IAM token that Aurora accepts as a password.

This page covers configuring the trust relationship and permissions between two roles that you own: the Connect role that your deployment runs as, and the database connect role in the Aurora account. The Connect role and its runtime binding (IRSA or an instance profile) must already exist.

After reading this page, you will be able to:

  • Create an IAM database connect role with the required permission policy and trust relationship

  • Grant your Redpanda Connect role cross-account access to Aurora

  • Configure a Redpanda Connect pipeline input to authenticate to Aurora using IAM roles

Prerequisites

  • Redpanda Connect deployed on AWS infrastructure that you manage, such as Amazon EKS or EC2

  • An Aurora cluster (PostgreSQL or MySQL)

  • Permissions to create IAM roles and attach inline policies in both the Connect and Aurora AWS accounts

  • An IAM role associated with the Redpanda Connect pod or EC2 instance (see Identify your Redpanda Connect role)

Identify your Redpanda Connect role

The Connect role is the IAM role that your Redpanda Connect process runs as:

  • On Amazon EKS, this is the IRSA role associated with the Kubernetes service account that runs your Redpanda Connect pods. To set up IRSA, see the Amazon EKS documentation.

  • On EC2, this is the role attached to the instance profile. To set up an instance profile, see the AWS IAM documentation.

The AWS SDK resolves this identity automatically through the default credential chain, so no credentials appear in your pipeline configuration.

Find the Aurora cluster resource ID

The policy for RDS IAM authentication requires the Aurora cluster resource ID (not the cluster identifier). Run the following command to retrieve the resource ID:

aws rds describe-db-clusters \
  --query "DBClusters[?DBClusterIdentifier=='<cluster-name>'].[DbClusterResourceId]" \
  --output text \
  --profile <db-account-aws-profile>

The resource ID has the format cluster-<alphanumeric-string>.

Create the database connect role

In the AWS account where Aurora is hosted, create an IAM role with the following permission policy:

permission-policy.json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "rds-db:connect",
            "Effect": "Allow",
            "Resource": "arn:aws:rds-db:<region>:<db-account-id>:dbuser:cluster-<resource-id>/<iam-db-user>"
        }
    ]
}

Replace the following placeholders:

  • <region>: The AWS region of the Aurora cluster

  • <db-account-id>: The AWS account ID that hosts Aurora

  • <resource-id>: The cluster resource ID from Find the Aurora cluster resource ID

  • <iam-db-user>: The database user configured for IAM authentication

Trust policy

Attach the following trust policy to the role to allow your Connect role to assume it:

trust-policy.json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<connect-account-id>:role/<connect-role-name>"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Replace the following placeholders:

The database connect role must be owned by the same AWS account as the Aurora instance. For cross-account setups, create this role in the Aurora account, not the account where Redpanda Connect runs.

Create the role

Run the following AWS CLI commands to create the role and apply the permission policy:

aws iam create-role \
  --role-name "<db-connect-role-name>" \
  --assume-role-policy-document file://trust-policy.json \
  --profile <db-account-aws-profile>

aws iam put-role-policy \
  --role-name "<db-connect-role-name>" \
  --policy-name "aurora-rds-connect" \
  --policy-document file://permission-policy.json \
  --profile <db-account-aws-profile>

Grant the Connect role cross-account access

In the AWS account where your Connect role lives, add an inline policy to the Connect role (identified in Identify your Redpanda Connect role) to allow it to assume your database connect role:

inline-policy.json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::<db-account-id>:role/<db-connect-role-name>"
        }
    ]
}

To attach the policy, run:

aws iam put-role-policy \
  --role-name "<connect-role-name>" \
  --policy-name "allow-x-account-db-connect" \
  --policy-document file://inline-policy.json \
  --profile <connect-account-aws-profile>

To verify the policy is attached, run:

aws iam get-role-policy \
  --role-name "<connect-role-name>" \
  --policy-name "allow-x-account-db-connect" \
  --profile <connect-account-aws-profile>

Configure the network

The Aurora security group must allow inbound traffic on the database port (5432 for PostgreSQL, 3306 for MySQL) from the source that Aurora sees:

  • For EKS or EC2 deployments in the same VPC as Aurora, allow the node or instance security group, or the private CIDR range that your Redpanda Connect pods or instances use.

  • For deployments that reach Aurora across a VPC boundary, such as through VPC peering or from another network, allow the IP addresses that your traffic originates from after translation, such as your NAT gateway IPs.

Configure the pipeline

In your Redpanda Connect pipeline YAML, set the aws.roles field to the ARN of the database connect role created in Create the database connect role.

This configuration applies to the postgres_cdc, pg_stream, and mysql_cdc inputs.

For the postgres_cdc and pg_stream inputs:

input:
  postgres_cdc:
    dsn: "host=<aurora-endpoint> port=5432 user=<iam-db-user> dbname=<db-name> sslmode=require"
    aws:
      enabled: true
      region: <region>
      endpoint: <aurora-endpoint>
      roles:
        - role: arn:aws:iam::<db-account-id>:role/<db-connect-role-name>

For the mysql_cdc input:

input:
  mysql_cdc:
    dsn: "<iam-db-user>@tcp(<aurora-endpoint>:3306)/<db-name>?tls=true"
    aws:
      enabled: true
      region: <region>
      endpoint: <aurora-endpoint>
      roles:
        - role: arn:aws:iam::<db-account-id>:role/<db-connect-role-name>

The aws.roles field accepts an array of role ARNs, which are assumed in sequence. This supports chaining multiple role assumptions for more complex cross-account setups.

Troubleshooting

These errors appear in the Redpanda Connect process logs.

is not authorized to perform: sts:AssumeRole

The inline assume-role policy is missing from your Connect role, or the trust policy on the database connect role does not name your Connect role.

  1. Verify that the database connect role’s trust policy lists your Connect role as a principal, as described in Trust policy.

  2. Attach the inline policy as described in Grant the Connect role cross-account access.

The error clears automatically after the policy is in place. Look for a message like postgres_cdc input go active or mysql_cdc input go active in the logs to confirm successful authentication.

Next steps