Cloud

Configure GCP Private Service Connect in the Cloud Console

  • This guide is for configuring GCP Private Service Connect using the Redpanda Cloud Console. To configure and manage Private Service Connect on an existing cluster with public networking, you must use the Cloud API for BYOC or the Cloud API for Dedicated.

  • The latest version of Redpanda GCP Private Service Connect (available March, 2025) supports AZ affinity. This allows requests from Private Service Connect endpoints to stay within the same availability zone, avoiding additional networking costs.

  • DEPRECATION: The original Redpanda GCP Private Service Connect is deprecated and will be removed in a future release. For more information, see Deprecated features.

The Redpanda GCP Private Service Connect service provides secure access to Redpanda Cloud from your VPC network. Traffic over Private Service Connect remains within GCP’s private network, avoiding the public internet. Your VPC network can access the Redpanda VPC network, but Redpanda cannot access your VPC network.

Consider using Private Service Connect if you have multiple VPC networks and could benefit from a more simplified approach to network management.

  • Each consumer VPC network can have one Private Service Connect endpoint connected to the Redpanda service attachment.

  • Private Service Connect allows overlapping CIDR ranges in VPC networks.

  • The number of connections is limited only by your Redpanda usage tier. Private Service Connect does not add extra connection limits.

  • You control from which GCP projects connections are allowed.

Prerequisites

  • Use the gcloud command-line interface (CLI) to create the consumer-side resources, such as a consumer VPC network and forwarding rule, or to modify existing resources to use the Private Service Connect service attachment created for your cluster.

  • The consumer VPC network must be in the same region as your Redpanda cluster.

Enable Private Service Connect for existing clusters

  1. In the Redpanda Cloud Console, open your cluster, and click Dataplane settings.

  2. Under Private Service Connect, click Enable.

  3. For the accepted consumers list, you need the GCP project IDs from which incoming connections will be accepted.

  4. It may take several minutes for your cluster to update. When the update is complete, the Private Service Connect status in Dataplane settings changes from In progress to Enabled.

Deploy consumer-side resources

For each consumer VPC network, you must complete the following steps to successfully connect to the service attachment and use the Kafka API and other Redpanda services, such as HTTP Proxy.

  1. In Dataplane settings, copy the DNS zone and Service attachment URL under Private Service Connect. Use this URL to create the Private Service Connect endpoint in GCP.

  2. Get the name of the consumer VPC network and the subnet <psc-endpoint-subnet>, where the Private Service Connect endpoint forwarding rule will be created.

  3. Create a Private Service Connect IP address for the endpoint:

    gcloud compute addresses create <psc-endpoint-ip-name> --subnet=<psc-endpoint-subnet> --addresses=<psc-endpoint-ip> --region=<region>

  4. Create the Private Service Connect endpoint forwarding rule:

    If you enabled global access when creating the cluster, you must include the --allow-psc-global-access flag to configure the endpoint to accept client connections from different regions. 
    gcloud compute forwarding-rules create <psc-endpoint-forwarding-rule-name> --region=<region> --network=<consumer-vpc-name> --address=<psc-endpoint-ip> --target-service-attachment=<rp-psc-service-attachment-url>

  5. Create firewall rules allowing egress traffic to the Private Service Connect endpoint:

    gcloud compute firewall-rules create redpanda-psc-egress \
  --description="Allow access to Redpanda PSC endpoint" \
  --network="<consumer-vpc-name>" \
  --direction="EGRESS" \
  --destination-ranges=<psc-endpoint-ip> \
  --allow="tcp:443,tcp:30081,tcp:30282,tcp:30292,tcp:32092-32141,tcp:35082-35131,tcp:32192-32241,tcp:35182-35231,tcp:32292-32341,tcp:35282-35331"

  6. Create a private DNS zone. Use the cluster DNS zone value as the DNS name:

    gcloud dns managed-zones create <dns-zone-name> \
  --project=<gcp-project-id> \
  --description="Redpanda Private Service Connect DNS zone" \
  --dns-name="<dns-zone-from-the-ui>" \
  --visibility="private" \
  --networks="<consumer-vpc-name>"

  7. In the newly-created DNS zone, create a wildcard DNS record using the cluster DNS record value:

    gcloud dns record-sets create '*.<dns-zone-from-the-ui>' \
  --project=<gcp-project-id> \
  --zone="<dns-zone-name>" \
  --type="A" \
  --ttl="300" \
  --rrdatas="<psc-endpoint-ip>"

Access Redpanda services through Private Service Connect endpoint

After you have enabled Private Service Connect for your cluster, your connection URLs are available in the How to Connect section of the cluster overview in the Redpanda Cloud UI.

You can access Redpanda services such as Redpanda Console, Schema Registry, and HTTP Proxy from the client VPC or virtual network; for example, from a compute instance in the VPC or network.

The bootstrap server hostname is unique to each cluster. The service attachment exposes a set of bootstrap ports for access to Redpanda services. These ports load balance requests among brokers. Make sure you use the following ports for initiating a connection from a consumer:

Redpanda service Default port

Kafka API

30292

HTTP Proxy

30282

Schema Registry

30081

Redpanda Console

443

Access Kafka API seed service

Use port 30292 to access the Kafka API seed service.

export RPK_BROKERS='<kafka-api-bootstrap-server-hostname>:30292'
rpk cluster info -X tls.enabled=true -X user=<user> -X pass=<password>

When successful, the rpk output should look like the following:

CLUSTER
=======
redpanda.rp-cki01qgth38kk81ard3g

BROKERS
=======
ID    HOST                                                                PORT   RACK
0*    0-3da65a4a-0532364.cki01qgth38kk81ard3g.fmc.dev.cloud.redpanda.com  32092  use2-az1
1     1-3da65a4a-63b320c.cki01qgth38kk81ard3g.fmc.dev.cloud.redpanda.com  32093  use2-az1
2     2-3da65a4a-36068dc.cki01qgth38kk81ard3g.fmc.dev.cloud.redpanda.com  32094  use2-az1

Access Schema Registry seed service

Use port 30081 to access the Schema Registry seed service.

curl -vv -u <user>:<password> -H "Content-Type: application/vnd.schemaregistry.v1+json" --sslv2 --http2 <schema-registry-bootstrap-server-hostname>:30081/subjects

Access HTTP Proxy seed service

Use port 30282 to access the Redpanda HTTP Proxy seed service.

curl -vv -u <user>:<password> -H "Content-Type: application/vnd.kafka.json.v2+json" --sslv2 --http2 <http-proxy-bootstrap-server-hostname>:30282/topics

Verify the Redpanda Console network path

When you configure private connectivity through the Cloud API, set connect_console: true on the network configuration to enable Console access through the cluster’s private endpoint. The Cloud UI sets this for you when you enable private connectivity.

The Redpanda Console URL is served on port 443 (HTTPS) and follows the form https://console-<id>.<cluster_domain>;. The <id> is a per-cluster suffix assigned by the Redpanda control plane and is opaque to you; the full URL is shown in the How to Connect section of the cluster overview in the Redpanda Cloud Console.

To verify that the network path to Redpanda Console is open, run the following commands from a host in the client network. First, confirm DNS resolves the hostname to a private IP on the cluster’s private endpoint:

dig +short console-<id>.<cluster_domain>

The response is a private IP from your client network’s address range, for example:

10.0.0.42

Then confirm Redpanda Console responds over HTTPS through the endpoint:

curl -sS -o /dev/null -w "%{http_code}\n" https://console-<id>.<cluster_domain>/

Expected output:

200

A 200 response confirms the network path. The Console UI itself does not expose a standalone login form: sign in to cloud.redpanda.com, navigate to the cluster, and use the cluster’s left navigation (Topics, Brokers, Consumer groups) to interact with Redpanda Console.

  • DNS resolution for the Redpanda Console hostname is handled automatically by the cluster’s private endpoint. You don’t need to create a private hosted zone or override DNS in the client network.

  • Ensure your network access rules (for example, AWS security groups, Azure NSGs, or GCP firewall rules) on the private endpoint allow inbound TCP on port 443 from your client workload sources only (for example, the client network’s CIDR or specific client access groups). Avoid broad source ranges such as 0.0.0.0/0.

Test the connection

You can test the Private Service Connect connection from any VM or container in the consumer VPC. If configuring a client isn’t possible right away, you can do these checks using rpk or curl:

  1. Set the following environment variables.

    export RPK_BROKERS='<kafka-api-bootstrap-server-hostname>:30292'
export RPK_TLS_ENABLED=true
export RPK_SASL_MECHANISM="<SCRAM-SHA-256 or SCRAM-SHA-512>"
export RPK_USER=<user>
export RPK_PASS=<password>

  2. Create a test topic.

    rpk topic create test-topic

  3. Produce to the test topic.

    • rpk

    • curl

    echo 'hello world' | rpk topic produce test-topic
    curl -s \
  -X POST \
  "<http-proxy-bootstrap-server-url>/topics/test-topic" \
  -H "Content-Type: application/vnd.kafka.json.v2+json" \
  -d '{
  "records":[
      {
          "value":"hello world"
      }
  ]
}'

  4. Consume from the test topic.

    • rpk

    • curl

    rpk topic consume test-topic -n 1
    curl -s \
  "<http-proxy-bootstrap-server-url>/topics/test-topic/partitions/0/records?offset=0&timeout=1000&max_bytes=100000"\
  -H "Accept: application/vnd.kafka.json.v2+json"

Disable Private Service Connect

In Dataplane settings, click Disable. Existing connections are closed after GCP Private Service Connect is disabled. To connect using Private Service Connect again, you must re-enable the service.