Create a BYOC Cluster on GCP

To create a Redpanda cluster in your virtual private cloud (VPC), follow the instructions in the Redpanda Cloud UI. The UI contains the parameters necessary to successfully run rpk cloud byoc apply. See also: BYOC architecture.

With standard BYOC clusters, Redpanda manages security policies and resources for your VPC, including subnetworks, service accounts, IAM roles, firewall rules, and storage buckets. For the highest level of security, you can manage these resources yourself with a BYOVPC cluster on GCP.

If your clients need to connect from different GCP regions than where your cluster will be deployed, you must enable global access during cluster creation using the Cloud API. To create a BYOC cluster with global access enabled, see Enable Global Access.

Create a BYOC cluster

  1. Log in to Redpanda Cloud.

  2. On the Clusters page, click Create cluster, then click Create BYOC cluster.

    Enter a cluster name, then select the resource group, provider (GCP), region, tier, availability, and Redpanda version.

    • If you plan to create a private network in your own VPC, select the region where your VPC is located.

    • Three availability zones provide two backups in case one availability zone goes down.

    Optionally, click Advanced settings to specify up to five key-value custom GCP labels. If a label key starts with gcp.network-tag.<tag>, then the agent interprets it as a request to apply the <tag> network tag to GCE instances in the cluster. Use labels for organization/metadata; use network tags to target firewall rules and routes. After the cluster is created, labels are applied to applicable GCP resources (for example, instances and disks), and network tags are applied to instances. For more information, see the GCP documentation. After the cluster is created, you can specify more labels with the Cloud API.

  3. Click Next.

  4. On the Network page, enter the connection type: either Public or Private. For BYOC clusters, Private is best-practice.

    • Your network name is used to identify this network.

    • For a CIDR range, choose one that does not overlap with your existing VPCs or your Redpanda network.

  5. Click Next.

  6. On the Deploy page, follow the steps to log in to Redpanda Cloud and deploy the agent.

    Note that rpk configures the permissions required by the agent to provision and actively maintain the cluster. For details about these permissions, see GCP IAM permissions.

Redpanda Cloud does not support customer access to the Kubernetes control plane with kubectl. This restriction allows Redpanda Data to manage all configuration changes internally to ensure a 99.99% service level agreement (SLA) for BYOC clusters.

Manage custom resource labels and network tags

Your organization might require custom resource labels and network tags for cost allocation, audit compliance, or governance policies. After cluster creation, you can manage this with the Cloud Control Plane API. The Control Plane API allows up to 16 custom resource labels and network tags in GCP.

Make sure you have:

  • The cluster ID. You can find this in the Redpanda Cloud UI, in the Details section of the cluster overview.

  • A valid bearer token for the Cloud Control Plane API. For details, see Authenticate to the API.

To unlock this feature for your account, contact Redpanda Support.

  1. To refresh agent permissions so the Redpanda agent can update labels and network tags, run:

    export CLUSTER_ID="<cluster-id>"
    export PROJECT_ID="<gcp-project-id>"
    
    rpk cloud byoc gcp apply --redpanda-id="$CLUSTER_ID" --project-id="$PROJECT_ID"

    This step is required because label/tag management requires additional IAM permissions that may not have been granted during initial cluster creation:

    • compute.disks.get

    • compute.disks.list

    • compute.disks.setLabels

    • compute.instances.setLabels

  2. To update labels and network tags, invoke the Cloud API.

    First, set your authentication token:

    export AUTH_TOKEN="<your-bearer-token>"

    The PATCH call sets the labels and network tags specified under "cloud_provider_tags". It replaces the existing labels and tags with the specified labels and tags. Include all desired labels and tags in the request. To remove a single entry, omit it from the map you send.

    cluster_patch_body=$(cat <<'JSON'
    {
      "cloud_provider_tags": {
        "environment": "production",
        "cost-center": "engineering",
        "gcp.network-tag.web-servers": "true",
        "gcp.network-tag.database-access": "true"
      }
    }
    JSON
    )
    
    curl -X PATCH "https://api.redpanda.com/v1/clusters/$CLUSTER_ID" \
       -H "Content-Type: application/json" \
       -H "Authorization: Bearer $AUTH_TOKEN" \
       -d "$cluster_patch_body"

    To remove all labels and network tags, send an empty cloud_provider_tags object:

    cluster_patch_body='{"cloud_provider_tags": {}}'
    
    curl -X PATCH "https://api.redpanda.com/v1/clusters/$CLUSTER_ID" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $AUTH_TOKEN" \
      -d "$cluster_patch_body"