Add a BYOC VPC Peering Connection on GCP

A VPC peering connection is a networking connection between two VPCs. This connection allows the VPCs to communicate with each other as if they were within the same network. A route table routes traffic between the two VPCs using private IPv4 addresses.

To start sending data to the Redpanda cluster, you must configure the VPC network connection by connecting your Redpanda VPC to your existing GCP VPC.

Prerequisites

Create VPCs

  1. Go to the VPC section in your GCP project UI.

  2. You should see an existing VPC. This has an ID with a redpanda- prefix.

  3. If you don’t already have a second VPC to connect your Redpanda network to, create one.

    • This is your Redpanda client. Ensure that its CIDR does not overlap with the Redpanda network from step 1.

    • The following example uses the name rp-client.

Create a new peering connection

  1. In the GCP project UI, go to Peering Connections.

  2. Create a new peering connection with the following values:

    • Your VPC network: rp-client

    • Peered VPC network: redpanda-<id>

  3. Save changes.

  4. Create another peering connection, with the reverse values as above:

    • Your VPC network: redpanda-<id>

    • Peered VPC network: rp-client

  5. Save changes. GCP should set up routing automatically.

Connect to Redpanda

The cluster Overview page has a variety of ways for you to connect and start sending data.

To quickly test this quickly in GCP:

  • Create a virtual machine on your GCP network that has a firewall rule allowing ingress traffic from your IP (for example, <your-ip>/32)

  • Activate the Cloud Shell in your project, install rpk in the Cloud Shell, and run rpk cluster info.

  • If there is output from Redpanda, your connection is successful.

Switch from VPC peering to Private Service Connect

VPC peering and Private Service Connect use the same DNS hostnames (connection URLs) to connect to the Redpanda cluster. When you configure the Private Service Connect DNS, those hostnames resolve to Private Service Connect endpoints, which can interrupt existing VPC peering-based connections if clients aren’t ready.

To enable Private Service Connect without disrupting VPC peering connections, do a controlled DNS switchover:

  1. Enable Private Service Connect on the existing cluster and deploy consumer-side resources, but do not create private DNS yet. See: Enable Private Service Connect on an existing cluster.

  2. During a planned window, create the private DNS zone and records in your VPC to switch the shared hostnames over to Private Service Connect.