Add a BYOC VPC Peering Connection on AWS

A VPC peering connection is a networking connection between two VPCs. This connection allows the VPCs to communicate with each other as if they were within the same network. A route table routes traffic between the two VPCs using private IPv4 addresses.

To start sending data to the Redpanda cluster, you must configure the VPC network connection by connecting your Redpanda VPC to your existing AWS VPC.

Prerequisites

Create a peering connection

  1. In the AWS management console or the CLI, create a new peering connection between your AWS VPC and your Redpanda network using the following:

    • VPC Requester: Your Redpanda VPC. This looks something like network-ch2c2ntioepec6ilaoog.

    • VPC Accepter: Your existing AWS VPC ID.

  2. After the VPC peering connection is created, make note of your peering connection ID. It has a pcx- prefix.

Create routes from Redpanda to AWS

The following command routes traffic from Redpanda to AWS by finding the route tables for each associated subnet and creating a route:

aws ec2 describe-route-tables --filter "Name=tag:Name,Values=network-<redpanda-network-id>" "Name=tag:purpose,Values=private" | jq -r '.RouteTables[].RouteTableId' | \
while read -r route_table_id; do \
aws ec2 create-route --route-table-id $route_table_id --destination-cidr-block <aws-vpc-cidr-block> --vpc-peering-connection-id <peering-connection-id>; \
done;

Replace the following placeholder values:

  • Redpanda network ID: This ID appears after clicking on the name of the Redpanda network in the Details section of the Overview page of your cluster. This network ID may look similar, however, it is distinct from your cluster ID.

  • AWS CIDR block: This is listed in the AWS UI Details for your VPC.

  • Peering connection ID: This is the ID of the peering connection noted in step one.

Create routes from AWS to Redpanda

Now you must route your AWS subnet(s) to your Redpanda CIDR. The base command:

aws ec2 --region <region> create-route \
  --route-table-id <aws-route-table-id> \
  --destination-cidr-block <redpanda-cidr-range> \
  --vpc-peering-connection-id <peering-connection-id>

Your VPC may have multiple subnets, which may have multiple route table associations. Add the route to all the subnets.

Test your connection

There are two ways to test your connection:

  • Return to your cluster overview, and follow the directions in the How to connect panel.

  • Use the AWS Reachability Analyzer. Select your VM instance and a Redpanda instance as the source and destination, and test the connection between them.

VPC peering and PrivateLink use the same DNS hostnames (connection URLs) to connect to the Redpanda cluster. When you configure the PrivateLink DNS, those hostnames resolve to PrivateLink endpoints, which can interrupt existing VPC peering-based connections if clients aren’t ready.

To enable PrivateLink without disrupting VPC peering connections, do a controlled DNS switchover:

  1. Enable PrivateLink on the existing cluster and configure the PrivateLink connection to Redpanda Cloud, but do not modify VPC DNS attributes yet. See: Enable PrivateLink on an existing cluster.

  2. During a planned window, modify the VPC DNS attributes to switch the shared hostnames over to PrivateLink.