Authenticate to Amazon Aurora using IAM roles
Use IAM Roles for Service Accounts (IRSA) to authenticate Redpanda Connect pipelines to Amazon Aurora on AWS without storing static credentials.
Authentication uses a two-hop role chain. The Redpanda Connect pod assumes a pipeline IAM role through IRSA. That role then assumes a database-specific role in your Aurora account, which generates a short-lived RDS IAM token that Aurora accepts as a password.
This page covers configuring both roles with the required trust relationships. Redpanda manages the pipeline role and you add a single inline policy to it. You create and own the database connect role.
After reading this page, you will be able to:
-
Create an IAM database connect role with the required permission policy and trust relationship
-
Grant the Redpanda Connect pipeline role cross-account access to Aurora
-
Configure a Redpanda Connect pipeline input to authenticate to Aurora using IAM roles
Prerequisites
-
Redpanda Connect deployed on AWS
-
An Aurora cluster (PostgreSQL or MySQL)
-
Permissions to create IAM roles and attach inline policies in both the Redpanda and Aurora AWS accounts
-
An IAM role associated with the Redpanda Connect pod (see Find the pipeline IAM role name)
Find the pipeline IAM role name
For Redpanda Cloud BYOC deployments, the pipeline role is pre-created in the AWS account where your cluster is deployed and follows this naming convention:
redpanda-<cluster-id>-redpanda-connect-pipeline
| Do not modify the existing policies attached to this role. Redpanda manages these policies and reverts manual changes automatically. To avoid conflicts, add only new inline policies to this role. |
For self-managed deployments, use the IAM role associated with your Redpanda Connect pod or EC2 instance.
Find the Aurora cluster resource ID
The policy for RDS IAM authentication requires the Aurora cluster resource ID (not the cluster identifier). Run the following command to retrieve the resource ID:
aws rds describe-db-clusters \
--query "DBClusters[?DBClusterIdentifier=='<cluster-name>'].[DbClusterResourceId]" \
--output text \
--profile <db-account-aws-profile>The resource ID has the format cluster-<alphanumeric-string>.
Create the database connect role
In the AWS account where Aurora is hosted, create an IAM role with the following permission policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "rds-db:connect",
"Effect": "Allow",
"Resource": "arn:aws:rds-db:<region>:<db-account-id>:dbuser:cluster-<resource-id>/<iam-db-user>"
}
]
}Replace the following placeholders:
-
<region>: The AWS region of the Aurora cluster -
<db-account-id>: The AWS account ID that hosts Aurora -
<resource-id>: The cluster resource ID from Find the Aurora cluster resource ID -
<iam-db-user>: The database user configured for IAM authentication
Trust policy
Attach the following trust policy to the role to allow the Redpanda pipeline role to assume it:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<rp-account-id>:role/<pipeline-role-name>"
},
"Action": "sts:AssumeRole"
}
]
}Replace the following placeholders:
-
<rp-account-id>: The AWS account ID where the Redpanda Connect pipeline role lives -
<pipeline-role-name>: The pipeline role name from Find the pipeline IAM role name
| The database connect role must be owned by the same AWS account as the Aurora instance. For cross-account setups, create this role in the Aurora account, not the Redpanda account. |
Required tag
The database connect role must have the following tag. Without it, the pipeline role cannot assume the database role.
| Key | Value |
|---|---|
|
|
Create the role
Run the following AWS CLI commands to create the role and apply the permission policy and tag:
aws iam create-role \
--role-name "<db-connect-role-name>" \
--assume-role-policy-document file://trust-policy.json \
--profile <db-account-aws-profile>
aws iam put-role-policy \
--role-name "<db-connect-role-name>" \
--policy-name "aurora-rds-connect" \
--policy-document file://permission-policy.json \
--profile <db-account-aws-profile>
aws iam tag-role \
--role-name "<db-connect-role-name>" \
--tags Key=redpanda_scope_redpanda_connect,Value=true \
--profile <db-account-aws-profile>Grant the pipeline role cross-account access
In the AWS account where the Redpanda Connect pipeline role lives, add an inline policy to the pipeline role (identified in Find the pipeline IAM role name) to allow it to assume your database connect role:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::<db-account-id>:role/<db-connect-role-name>",
"Condition": {
"StringEquals": {
"aws:ResourceTag/redpanda_scope_redpanda_connect": "true"
}
}
}
]
}To attach the policy, run:
aws iam put-role-policy \
--role-name "<pipeline-role-name>" \
--policy-name "allow-x-account-db-connect" \
--policy-document file://inline-policy.json \
--profile <redpanda-aws-profile>To verify the policy is attached, run:
aws iam get-role-policy \
--role-name "<pipeline-role-name>" \
--policy-name "allow-x-account-db-connect" \
--profile <redpanda-aws-profile>Configure the network
The Aurora security group must allow inbound traffic from your Redpanda Connect cluster’s outbound IP addresses on the database port (5432 for PostgreSQL, 3306 for MySQL).
For Redpanda Cloud BYOC deployments, retrieve the NAT Gateway IPs using the Redpanda Cloud API:
AUTH_TOKEN=$(curl -s --request POST \
--url 'https://auth.prd.cloud.redpanda.com/oauth/token' \
--header 'content-type: application/x-www-form-urlencoded' \
--data grant_type=client_credentials \
--data client_id="${REDPANDA_CLIENT_ID}" \
--data client_secret="${REDPANDA_CLIENT_SECRET}" \
--data audience=cloudv2-production.redpanda.cloud \
| jq -r '.access_token')
curl -s -X GET "https://api.cloud.redpanda.com/v1/clusters/${RP_CLUSTER_ID}" \
-H "Authorization: Bearer ${AUTH_TOKEN}" \
| jq .cluster.nat_gatewaysConfigure the pipeline
In your Redpanda Connect pipeline YAML, set the aws.roles field to the ARN of the database connect role created in Create the database connect role.
This configuration applies to the postgres_cdc, pg_stream, and mysql_cdc inputs.
For the postgres_cdc and pg_stream inputs:
input:
postgres_cdc:
dsn: "host=<aurora-endpoint> port=5432 user=<iam-db-user> dbname=<db-name> sslmode=require"
aws:
enabled: true
region: <region>
endpoint: <aurora-endpoint>
roles:
- role: arn:aws:iam::<db-account-id>:role/<db-connect-role-name>For the mysql_cdc input:
input:
mysql_cdc:
dsn: "<iam-db-user>@tcp(<aurora-endpoint>:3306)/<db-name>?tls=true"
aws:
enabled: true
region: <region>
endpoint: <aurora-endpoint>
roles:
- role: arn:aws:iam::<db-account-id>:role/<db-connect-role-name>The aws.roles field accepts an array of role ARNs, which are assumed in sequence. This supports chaining multiple role assumptions for more complex cross-account setups.
Troubleshooting
These errors appear in Redpanda Connect pipeline logs. Monitor logs using rpk topic consume __redpanda.connect.logs.
is not authorized to perform: sts:AssumeRole
The inline assume-role policy is missing from the Redpanda Connect pipeline role, or the tag condition is not met.
-
Verify that the database connect role has the
redpanda_scope_redpanda_connect: truetag. -
Attach the inline policy as described in Grant the pipeline role cross-account access.
The error clears automatically after the policy is in place. Monitor pipeline logs to confirm:
rpk topic consume __redpanda.connect.logs --offset end | grep '<pipeline-id>'Look for a message like postgres_cdc input go active or mysql_cdc input go active to confirm successful authentication.
